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Preface 


The  purpose  of  this  thesis  was  to  collect  basic  information  on  the  statistical 
concept  base  potential  GSM  students  usually  bring  to  AFIT  and  to  determine 
which  statistical  concepts  are  used  by  acquisition  managers.  The  specific  appli¬ 
cation  of  Ausubel’s  Learning  Theory  and  Johnson’s  Curriculum  Model  was  to 
statistics,  even  though  both  can  be  applied  to  other  disciplines. 

In  performing  the  research  and  writing  of  this  thesis  I  have  received  a  great 
deal  of  help  and  support  from  others.  I  aim  deeply  indebted  to  my  advisor,  Dan 
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manning  the  AFIT  computer  centers  for  putting  up  with  my  endless  computer 
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and  concern  on  those  many  nights  when  I  was  else  where  with  thesis  work. 
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Abstract 


The  purpose  of  this  thesis  was  to  collect  basic  information  on  the  statistical 
concept  base  potential  GSM  students  usually  bring  to  AFIT  and  to  determine 
which  statistical  concepts  axe  used  by  acquisition  managers.  The  specific  appli¬ 
cation  of  Ausubel’s  Learning  Theory  and  Johnson’s  Curriculum  Model  was  to 
statistics,  even  though  both  can  be  applied  to  other  disciplines. 

In  order  to  accomplish  this  purpose  the  following  objectives  were  developed: 
(1)  to  collect  data  on  statistical  background  and  work  experience  of  acquisition 
managers  so  an  inference  could  be  made  concerning  the  statistical  concept  base 
of  entering  GSM  students;  (2)  to  determine  the  job  functions  being  performed  by 
acquisition  managers  so  that  problems  and  exercises  for  statistical  courses  can  be 
developed  to  represent  the  types  of  situations  GSM  students  can  be  expected  to 
encounter  upon  graduation  from  AFIT. 

Three  conclusions  were  reached  and  these  were;  (1)  while  there  are  indica¬ 
tions  that  statistics  is  needed  by  acquisition  managers,  the  current  survey  results 
show  that  the  acquisition  managers  are  not  using  any  of  the  identified  statistical 
course/concepts  in  their  work;  (2)  while  the  top  ranked  task  functions  axe  pri¬ 
marily  quantitative  in  nature,  the  courses  most  used  axe  qualitative  with  little  or 
no  quantitative  nature;  (3)  while  the  response  was  good  to  the  survey,  the  use  of 
this  survey  to  try  and  infer  the  concepts  entering  acquisition  management  GSM 
students  would  bring  to  AFIT  produced  unclear  results.  Any  attempt  to  infer  an 
association  among  the  statistical  concepts  with  the  activity  job  groups  will  most 
likely  require  personal  interviews  of  selected  field  personnel. 
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AN  INVESTIGATION  OF  THE  FACTORS 
MOTIVATING  MEANINGFUL  LEARNING  OF 
STATISTICS  BY  GRADUATE  SYSTEMS 
MANAGEMENT  STUDENTS  AT  AFIT 


I.  Introduction 


General  Issue 

Faced  with  acquiring  large  and  complex  systems  for  the  national  defense,  the 
Department  of  Defense  (DOD)  has  committed  vast  resources  to  the  implementa¬ 
tion  of  project  management  as  the  most  viable  and  effective  method  of  managing 
the  procurement  of  weapon  systems.  As  part  of  the  overall  implementation  plan, 
the  Air  Force  Institute  of  Technology  (AFIT)  has  been  tasked  by  HQ  USAF  to  pro¬ 
vide  the  systems  management  student  with  an  education  that  meets  the  needs  of 
the  Air  Force.  As  a  result,  AFIT’s  coordinator  for  the  Graduate  Systems  Manage¬ 
ment  (GSM)  Program  is  charged  with  providing  a  curriculum  that  is  responsive 
to  field  requirements  and  which  encourages  the  introduction  of  state-of-the-art 
problem  solving  technologies  in  the  classroom.  AFIT’s  teaching  staff’s  greatest 
challenge,  therefore,  is  to  foster  a  meaningful  learning  environment  that  motivates 
students  to  study  technology,  in  particular  quantitative  technologies,  and  to  apply 
these  upon  graduation  from  AFIT. 

In  his  book,  Educational  Psychology:  A  Cognitive  View ,  David  P.  Ausubel 
presents  a  theory  of  learning  that  declares  meaningful  learning  can  take  place: 

(1)  if  the  student  employs  a  meaningful  learning  set  (a  disposition  to 
relate  new  learning  material  meaningfully  to  his  existing  structure  of 
knowledge),  and  (2)  if  the  learning  task  itself  is  potentially  meaningful 
(if  it  itself  consists  of  plausible  and  sensible  material  and  if  it  can 
be  related  in  a  nonarbitrary  and  substantive  fashion  to  the  particular 
student’s  cognitive  structure)  (2:4). 
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Meaningful  learning  of  quantitative  material,  especially  statistical  material,  will 
take  place  at  AFIT,  if  courses  are  designed  to  (1)  accommodate  students’  need 
to  link  new  statistical  concepts  to  those  they  bring  with  them  and  (b)  foster 
a  meaningful  learning  set  by  introducing  applications  of  statistics  that  students 
perceive  as  useful. 

Previous  research  confirmed  the  utility  of  the  statistical  courses  taken  by 
GSM  students  (13:31),  but  failed  to  uncover  any  evidence  of  wide  spread  use  of 
statistics  by  the  27XX’s  in  the  field  (8:77).  Neither  Speck  nor  Koble  attempted 
to  identify  the  relationship  between  specific  statistical  techniques  and  job  tasks 
assigned  to  27XX  personnel.  To  date,  no  research  has  been  carried  out  to  assess 
what  statistical  concepts  the  entering  GSM  students  can  be  expected  to  possess 
(8;  13). 

If  GSM  students  are  to  encounter  learning  tasks  that  are  potentially  mean¬ 
ingful  and  a  learning  environment  that  fosters  a  meaningful  learning  set.  The 
statistical  concept  base  they  bring  to  AFIT  and  the  relationship  statistical  tech¬ 
nologies  maintain  with  critical  acquisition  functions,  especially  those  performed 
by  27XXs,  must  be  ascertained. 

Problem  Statement 

The  problem  is,  at  AFIT;  (1)  there  is  no  formal  procedure  to  assess  the 
statistical  maturity  of  entering  GSM  students,  and  (2)  there  is  a  general  lack  of 
knowledge  concerning  how  statistical  technology  is  or  could  be  related  to  tasks 
performed  by  practicing  acquisition  managers. 

Justification 

AFIT  has  provided  selected  officers  and  Air  Force  civilians  undergraduate 
and  graduate  level  education  since  1954  (15:2).  Longstanding  tradition  and  past 


results  require  AFIT  to  produce  highly  educated  personnel  to  perform  the  tasks  of 
acquiring  new  weapon  systems  for  use  by  the  operational  commands  of  the  United 
States  Air  Force.  Recently,  the  management  systems  created  for  such  procurement 
activities  have  come  under  attack  because  “weapon  systems  take  too  long  and  cost 
too  much  to  produce”  (12:xxii).  Clearly,  a  “better  job  of  determining  requirements 
and  estimating  costs  [is]  needed  at  the  outset  of  weapons  development”  (12:xxiii). 

In  order  to  meet  this  requirement  the  Air  Force  has  instituted  a  manage¬ 
ment  discipline  known  as  project  management.  This  initiative  has  received  the 
support  of  the  Blue  Ribbon  Commission  on  Defense  Management,  which  proposed 
the  establishment  of  Centers  of  Excellence.  Such  Centers  of  Excellence  are  popu¬ 
lated  by  project  teams  that  have  been  commended  for  their  ability  to  develop  and 
produce  new  weapons  systems  “rapidly,  efficiently,  and  with  high  quality  perfor¬ 
mance”  (12:42).  The  fact  is,  however,  this  form  of  project  management  requires 
the  project  managers  to  be  very  knowledgeable  in  project  performance  and  pos¬ 
sess  the  ability  to  recognize  immediately  when  a  problem  requires  their  attention 
(3:365;  1:34-45;  9:21-22;  12).  Such  skills  can  only  be  exercised  by  someone  who 
has  acquired  and  mastered  a  broad  range  of  analytical  skills,  including  the  corpus 
of  concepts  identified  within  the  field  of  applied  statistics. 

Confirmation  of  this  fact  was  obtained  by  Captain  Roger  D.  Koble  in  his  MS 
Thesis,  Applications  of  Computers  as  an  Aid  to  Decision-Making  in  Air  Force  Sys¬ 
tem  Program  Offices ,  Koble  found  that  project  managers  routinely  needed  to  know 
how  to  establish  sampling  schemes,  to  estimate  system  performance  parameters, 
and  to  forecast  future  procurement  activities.  It  appears  such  statistical  tools 
are  the  sine  qua  non  for  evaluating  costs,  schedules  and  technical  performance  of 
projects  (8:28-30).  Lt  Ernest  E.  Speck  in  his  MS  Thesis,  Perceived  Utility  of  the 
AFIT  Graduate  Systems  Management  Program ,  after  sampling  AFIT  graduates, 
found  57.9  percent  of  the  respondents  stated  they  definitely  needed  managerial 
statistics  on  the  job.  Most,  80.3  percent,  felt  the  level  of  presentation  of  statistics 


at  AFIT  was  correct.  More  importantly,  67.6  percent  stated  they  felt  more  prac¬ 
tical  applications  of  statistical  technology,  directly  related  to  the  job  functions  of 
AFIT  graduates,  should  be  included  in  the  AFIT  statistical  curriculum.  Interest¬ 
ingly  enough,  few  respondents,  16.2  percent,  felt  more  theory  should  be  included 
in  coursework  (13:31). 

Such  research  findings  lead  one  to  believe  acquisition  managers  have  a  grow¬ 
ing  requirement  for  practical  applications  of  statistics  in  the  education  and  train¬ 
ing.  Indeed,  the  definition  of  statistics  provided  by  Sam  Kash  Kachigan  in  his 
book,  Statistical  Analysis:  An  Interdisciplinary  Introduction  to  Univariate  &  Mul¬ 
tivariate  Methods ,  almost  sounds  like  the  job  description  of  an  acquisition  manager. 
He  proposes  the  following  definition  for  statistics: 

Observations  of  the  world  are  converted  into  numbers,  the  numbers 
are  manipulated  and  organized,  and  then  the  results  are  interpreted 
and  translated  back  to  a  world  that  is  now  hopefully  more  orderly 
and  understandable  than  prior  to  the  data  analysis.  This  process  of 
drawing  conclusions  and  understanding  more  about  the  sources  of  our 
data  is  the  goal  of  statistical  analysis  in  its  broadest  sense. 

More  specifically,  we  can  view  the  data  manipulation  and  organization 
as  achieving  one  or  more  of  three  basic  objectives:  (1)  data  reduction , 

(2)  service  as  an  inferential  measuring  tool,  and  (3)  the  identification 
of  associations  or  relationships  between  and  among  sets  of  data  (7:5). 


Acquisition  managers  have  responsibilities  for  meeting  all  three  objectives.  Quan¬ 
tifying  project  activities  for  purposes  of  data  reduction  and  display  Me  part  of 
their  daily  routine.  Making  inferences  about  system  parameters  based  on  day  to 
day  inputs  from  other  project  personnel  and  trying  to  identify  causal  relation¬ 
ships  among  variables  that  create  the  problems  of  weapons  system  acquisition  atre 
routine  acts  for  every  acquisition  manager  (3;  1;  9;  12). 

If  the  tasks  performed  by  the  acquisition  managers  can  be  cleaurly  linked  to 
specific  statisticad  analysis  techniques,  and  the  statistical!  concept  baise  required  to 
meet  adl  three  objectives  in  the  field  can  be  identified,  scenarios  can  be  constructed 
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that  will  be  perceived  as  useful  by  AFIT’s  GSM  students.  Furthermore,  if  job 
functions  are  clearly  documented  and  their  relationship  to  specific  statistical  tech¬ 
nologies  is  identified,  AFIT  faculty  will  have  a  much  clearer  understanding  of  the 
statistical  knowledge  base  entering  students  can  be  expected  to  bring  with  them 
to  AFIT.  Both  of  these  posibilities  served  to  motivate  the  goals  of  this  study:  to 
determine  the  functions  performed  by  acquisition  managers  and  to  assess  the  sta¬ 
tistical  background  of  entering  students  in  order  to  facilitate  a  meaningful  learning 
environment  for  statistics. 

Research  Objectives 

In  order  to  identify  the  statistical  concepts  entering  students  can  be  expected 
to  possess  and  in  order  to  facilitate  an  environment  for  meaningful  learning  this 
thesis  sought  to  meet  two  objectives: 

Objective  1.  To  collect  data  on  statistical  background  and  work  experience 
of  acquisition  managers,  so  an  inference  could  be  made  concerning  the  statistical 
concept  base  of  entering  GSM  students. 

Objective  2.  To  determine  the  job  functions  being  performed  by  acquisition 
managers,  so  that  problems  and  exercises  for  statistical  courses  can  be  developed 
to  represent  the  types  of  situations  GSM  students  can  be  expected  to  encounter 
upon  graduation  from  AFIT. 

Research  Questions 

To  meet  objective  1  the  following  research  questions  were  addressed: 

1.  What  coursework  and,  more  specifically,  what  exposure  to  statistical  con¬ 
cepts  via  academic  and/or  Professional  Continuing  Education  study  have  potential 


GSM  students  received? 


2.  What  statistical  concepts  have  found  application  in  the  environment  of 
potential  GSM  students? 

To  meet  objective  2  the  following  research  questions  were  addressed: 

3.  What  particular  job  functions  are  currently  being  accomplished  by  Ac¬ 
quisition  Managers? 

4.  Given  a  particular  job  function,  what  statistical  technologies  are  being 
used  to  help  solve  management  problems  associated  with  that  job  function? 

Scope 

This  research  studied  the  population  of  systems  acquisition  managers  in  the 
United  States  Air  Force.  Data  pertinent  to  this  study  were  collected  from  a  sample 
of  personnel  in  the  field  of  system  acquisition  management.  These  personnel  were 
selected  only  from  military  systems  acquisition  managers,  AFSCs  2716  and  2724. 
The  population  of  interest  is  the  military  systems  acquisition  managers,  AFSCs 
2716  and  2724,  with  and  without  an  AFIT  Systems  Management  Degree.  The 
sampling  plan  involved  the  use  of  a  stratified  sampling  model.  The  survey  was 
structured  to  collect  data  on  job  functions  performed  and  statistical  technologies 
used  by  any  of  the  strata  identified.  It  gathered  information  from  individuals  from 
the  rank  of  2nd  Lieutenant  (2d  Lt)  to  Colonel  (Col).  The  data  were  analyzed  for 
current  and  possible  applications  of  statistical  technologies. 

Limitations 

This  study  did  not  include  data  from  any  source  outside  of  the  system  acqui¬ 
sition  management  career  field  nor  information  from  individuals  below  the  rank 
of  2d  Lt  or  above  Col.  In  addition,  only  individuals  within  the  continental  United 
States  (CONUS)  were  surveyed. 
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Assumptions 

It  was  assumed  that  system  acquisition  managers  had  a  basic  knowledge  of 
system  management  tools  find  understood  how  to  apply  them  to  their  particular 
job.  Since  all  officers  hold  at  least  a  bachelors  degree,  and  only  replies  from  indi¬ 
viduals  who  had  at  least  six  months  experience  in  systems  acquisition  management 
were  retained,  this  assumption  seemed  to  be  reasonable. 

Definitions 

Cognitive  Structure.  Cognitive  structure  is  “the  total  content  and  organi¬ 
zation  of  a  given  individual’s  ideas;  or,  in  the  context  of  subject-matter  learning, 
the  content  and  organization  of  his  or  her  ideas  in  a  particular  area  of  knowledge” 
(2:625). 

Concepts.  Concepts  are  “objects,  events,  situations,  or  properties  that  pos¬ 
sess  common  criterial  attributes  (despite  diversity  along  other  dimensions  or  at¬ 
tributes)  and  are  designated  by  some  sign  or  symbol,  typically  a  word  with  generic 
meaning”  (2:625). 

Curriculum.  Curriculum  is  “a  logically  connected  set  of  conceptually  and 
pedagogically  analyzed  knowledge  and  value  claims”  (6:109). 

Discovery  Learning.  Discovery  learning  is  “that  kind  of  learning  in  which 
the  principal  content  of  what  is  to  be  learned  is  not  given  (or  presented),  but 
must  be  discovered  by  the  learner  before  he  can  assimilate  it  into  his  cognitive 
structure”  (2:626). 
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Educating.  Educating  is  the  process  of  “changing  the  meaning  of  human 
experience”  (6:124). 

Governance.  Governance  is  “the  power  in  a  social  setting  which  is  required 
to  bring  together  teaching,  curriculum,  and  learning.  Governance  controls  the 
meaning  that  controls  the  effort”  (6:153-154). 

Learning.  Learning  is  the  process  of  “the  active  reorganization  of  an  existing 
pattern  of  meaning”  (6:124). 

Meaningful  Learning.  Meaningful  learning  is  “the  acquisition  of  new  mean¬ 
ings;  it  presupposes  a  meaningful  learning  set  and  a  potentially  meaningful  learn¬ 
ing  task  (that  is,  a  task  that  can  be  related  in  nonarbitrary,  substantive  fashion 
to  what  the  learner  already  knows)”  (2:628). 

Meaningful  Learning  Set.  Meaningful  learning  set  is  “a  ‘disposition’  on  the 
part  of  a  learner  to  relate  a  learning  task  nonarbitrarily  and  substantively  to 
relevant  aspects  of  his  or  her  cognitive  structure”  (2:628). 

Progressive  Differentiation.  Progressive  differentiation  is  a  “part  of  the  pro¬ 
cess  of  meaningful  learning,  retention,  and  organization  that  results  in  further 
hierarchical  elaboration  of  concepts  or  propositions  in  cognitive  structure  from 
‘the  top  downwards’”  (2:629). 

Program  Manager  (PM).  The  single  Air  Force  manager  (system  program 
director,  program  or  project  manager,  or  system  or  item  manager)  during  any 
specific  phase  of  the  acquisition  life  cycle  (14:2). 
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Reception  Learning.  Reception  learning  is  “that  kind  of  learning  in  which 
the  entire  content  of  what  is  to  be  learned  is  presented  to  the  learner  in  more  or 
less  final  form”  (2:629). 

Rote  Learning.  Rote  learning  is  “the  acquisition  of  arbitrary,  verbatim  as¬ 
sociations  in  learning  situations  where  either  the  learning  material  itself  cannot  be 
nonarbitrarily  and  substantively  related  to  cognitive  structure  (that  is,  does  not 
possess  ‘logical  meaning’)  or  where  the  learner  exhibits  a  nonmeaningful  learning 
set”  (2:629). 

Subsumptive  Learning.  Subsumptive  learning  is  when  “learning  the  meaning 
of  a  new  concept  or  proposition  that  can  be  subsumed  under  a  relevant,  more  in¬ 
clusive  particular  idea(s)  in  cognitive  structure;  includes  derivative  and  correlative 
subsumption”  (2:630). 

Teaching.  Teaching  is  the  process  of  “achievement  of  shared  meaning  in  the 
context  of  educating”  (6:62). 

The  Four  Commonplaces  of  Educating.  The  four  commonplaces  of  educat¬ 
ing  are  teaching,  learning,  curriculum,  and  governance.  These  are  the  processes, 
expectations,  and  controls  of  the  educating  experience  (6:25). 

Overview  of  Thesis 

Chapter  II  will  summarize  the  theoretical  basis  for  the  survey  conducted  by 
this  research.  Chapter  III  will  outline  the  methodology  used  to  collect  data  needed 
to  answer  the  research  questions  posed  by  the  thesis.  Chapter  IV  will  present 
survey  results  concerning  the  job  functions  performed  by  acquisition  managers  and 
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make  inferences  about  the  statistical  concept  base  GSM  students  can  be  expected 
to  bring  with  them  to  AFIT.  Chapter  V  will  state  the  conclusions  of  the  thesis 
and  make  specific  recommendations  for  future  research. 
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II.  A  Theoretical  Model  for  Curriculum  and  Instructional  Design 
Introduction 

Establishing  a  meaningful  learning  environment  ultimately  requires  decisions 
be  made  concerning  what  is  to  be  taught  and  how  it  is  to  be  taught.  Educators 
must  answer  these  two  questions  when  performing  the  two  major  tasks  of  any 
course  design:  Curriculum  Development  and  Instructional  Planning.  When  the 
time  came  to  develop  a  methodology  to  answer  the  research  questions  posed  in 
Chapter  I  of  this  thesis,  it  became  obvious  that  a  model  would  be  required  to 
guide  the  development  of  the  survey  questionnaire. 

Model  Selection 

After  a  review  of  many  learning  theories  and  models  for  course  development, 
the  model  for  Curriculum  Development  and  Instruction  proposed  by  Mauritz  John¬ 
son  in  a  seminal  paper  entitled  Definitions  and  Models  in  Curriculum  Theory  (10) 
was  selected.  Figure  1  displays  the  fundamental  components  of  the  model  and 
provides  a  clear  indication  of  the  centred  roles  played  by  what  Johnson  labels  the 
Curriculum  Development  System  and  the  Instructional  System. 

For  purposes  of  this  research,  attention  was  focused  on  the  boxes  labeled 
Selection  Criteria  and  Ordering  Criteria  and  the  two  cells  dealing  with  the  product 
of  Curriculum  Development,  the  structured  series  of  intended  learning  outcomes 
(ILOs)  and  the  product  of  Instructional  Planning,  the  Instructional  Plan.  ILOs 
were  conceived  of  as  a  hierarchy  of  progressively  differentiated  concepts,  implying 
that  the  curriculum  should  address  the  most  general  concepts  of  statistics  first, 
and  deal  with  the  more  refined  notions  embedded  within  each  later. 

Although  Johnson  does  not  specify  the  form  of  ILO’s,  Ausbel’s  the¬ 
ory  would  indicate  that  these  should  be  concepts  to  be  learned,  for 
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(11:132) 

Figure  1:  Johnson’s  (1967)  model  for  curriculum  and  instruction 


with  them  we  effect  meaningful  learning.  In  other  words,  Johnson’s 
“curriculum  matrix”  produced  by  the  curriculum  development  system 
should  be  a  matrix  of  concepts.  To  the  extent  possible,  this  matrix 
should  suggest  hierarchical  and  subordinate  relationships  between  con¬ 
cepts,  although  this  feature  is  in  part  confounded  with  the  sequence  in 
which  concepts  are  taught  and  the  specific  exemplars  used  in  instruc¬ 
tion.  Skills,  attitudes,  and  values  should  be  considered  especially  as 
they  bear  on  learning  of  the  concepts  specified  (11:138). 


Development  of  a  revised  statistics  curriculum  mandates  careful  selection 
of  concepts  to  be  taught  and  formal  assessment  of  linkages  between  the  specific 
aspects  of  the  discipline  of  statistics  and  the  real  world  activities  of  GSM  students. 
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This  requirement  provided  the  rational  for  posing  the  first  two  research  questions 
of  this  thesis  and,  more  generally,  for  proposing  the  two  objectives  of  the  research 
effort:  to  acquire  data  on  the  statistical  background  of  entering  students  and  to 
determine  the  major  job  functions  performed  by  practicing  acquisition  managers. 
Unless  the  management  culture  of  the  entering  GSM  student  is  clearly  defined,  it 
will  be  impossible  to  divine  what  type  of  conceptual  base  he  can  be  expected  to 
bring  to  AFIT  or  what  type  of  concepts  should  be  taught  once  he  arrives.  Selection 
criteria  proposed  by  Johnson’s  model  document  the  need  to  carefully  choose,  from 
a  vast  array  of  concepts,  those  concepts  that  will  satisfy  GSM  students’  desire  for 
new  material  as  well  as  for  material  that  takes  into  consideration  the  conceptual 
base  each  student  brings  to  class.  In  Ausubelian  terms,  only  if  the  new  concepts 
to  be  taught  find  subsumption  within  the  more  familiar  cognitive  structure  of  the 
student  can  any  schooling  begin  to  facilitate  the  criteria  established  by  Ausubel 
in  the  quote  of  Chapter  I  for  “potentially  meaningful  learning  tasks.”  Indeed, 
one  of  the  things  that  made  Johnson’s  model  so  attractive  was  that  it  graphically 
inspires  one  to  fulfill  David  P.  Ausubel’s  major  dictum: 

If  I  had  to  reduce  all  of  educational  psychology  to  just  one  principle, 

I  would  say  this:  The  most  important  single  factor  influencing  learn¬ 
ing  is  what  the  learner  already  knows.  Ascertain  this  and  teach  him 
accordingly  (2:preface). 

The  long  term  goal  of  several  ongoing  research  efforts  is  to  provide  a  formal 
matrix  of  specific  statistical  concepts  for  the  curriculum  of  GSM  students  and 
to  develop  the  instructional  materials  that  foster  applications  of  statistics  that 
can  be  perceived  as  useful  by  GSM  students.  Thus,  it  was  very  reassuring  to 
read  in  Joseph  Novak’s  book,  A  Theory  of  Education ,  the  following  endorsement 
for  Johnson’s  model’s  ability  to  facilitate  completion  of  the  goad  of  this  thesis 
effort:  to  build  a  data  base  that  can  provide  the  information  required  to  select  the 
concepts  of  statistics  to  be  taught  and  to  build  the  scenarios  that  will  illustrate 
the  application  of  statistics  in  acquisition  management. 


If  learning  is  to  be  meaningful,  then  new  knowledge  to  be  learned  must 
have  relevant  anchoring  concepts  available  in  the  learner’s  cognitive 
structure.  Since  an  enormous  array  of  information  is  to  be  learned  in 
any  discipline,  only  the  most  general,  most  inclusive  concepts  are  likely 
to  provide  anchorage  in  a  wide  variety  of  learning  situations.  Johnson’s 
stress  on  selection  criteria  for  curriculum  planning  can  be  related  to 
Ausubel’s  stress  on  subsumption,  starting  with  the  most  general,  most 
inclusive  concepts.  And  Johnson’s  equal  emphasis  on  ordering  criteria 
can  be  linked  to  Ausubel’s  description  of  progressive  differentiation  of 
concepts  in  cognitive  structure  (10:137). 

The  Ultimate  Goal:  Meaningful  Reception  Learning 

Before  moving  to  Chapter  III  and  a  full  discussion  of  the  methodology  pro¬ 
posed  to  meet  the  objectives  of  this  research  effort  Johnson’s  model,  and  indeed, 
the  whole  effort  of  curriculum  development  and  instructional  planning,  need  to  be 
seen  as  merely  one  facet  of  the  educating  process.  It  seems  the  best  way  to  do  this 
is  to  point  out  that,  while  this  thesis  was  able  to  address  problems  encountered  in 
three  out  of  the  four  commonplaces  of  education:  Teaching,  Learning,  and  Cur¬ 
riculum,  the  fourth:  Governance,  which  may  be  the  most  important  factor  of  all 
in  trying  to  obtain  a  meaningful  learning  environment ,  was  simply  inaccessible  to 
influence  or  investigation.  Most  importantly,  the  entire  effort  was  conducted  in 
light  of  the  modes  of  learning  proposed  by  David  Ausubel. 

Dimensions  of  Learning 

Ausubel  declares  that  two  independent  dimensions  of  learning  create  an 
opportunity  for  four  different  types  of  learning  experience  for  the  student.  Figure  2 
lays  out  these  dimensions  graphically. 

Whereas  meaningful  learning  implies  subsumption  of  new  concepts  by  con¬ 
cepts  in  a  students  current  cognitive  structure,  rote  learning  leads  to  arbitrary 
inclusion  of  such  concepts.  Whereas,  reception  learning  implies  receiving  ex- 
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Figure  2:  Dimensions  of  Learning 


(20:146) 


clusively  from  someone  else,  discovery  learning  involves  becoming  aware  of  a 
concept  or  major  insight  without  outside  help.  Although  there  are  four  ex¬ 
treme  combinations:  ROTE/RECEPTION,  ROTE/DISCOVERY,  MEANING¬ 
FUL/RECEPTION,  and  MEANINGFUL/DISCOVERY.  Ausubel  makes  it  very 
clear  which  he  believes  takes  priority  in  the  classroom: 

As  shown  in  Figure  [2],  neither  meaningful  nor  discovery  learning  is 
an  absolute.  Rather,  each  can  be  located  on  the  rote- meaningful  and 
reception-discovery  continua.  For  logistical  reasons,  most  classroom 
learning,  especially  in  older  pupils,  is  meaningful  reception  learning. 
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However,  for  certain  kinds  of  learning,  and  in  younger  learners,  some 
degree  of  rote  and  discovery  learning  is  indicated  . . .  (2:4). 


Summary 

At  AFIT,  it  appears  that  because  of  time  constraints  and  an  enormous  course 
overload  most  learning  is  rote/reception.  Numerous  testimonies  of  students  and 
faculty  reveal  that  while  the  desire  is  present  on  the  part  of  both  faculty  and 
students  for  elevation  to  a  meaningful/reception  mode  during  classroom  periods 
and  meaningful /discovery  mode  during  research  quarters,  the  domination  of  the 
Governance,  which  so  far  chooses  to  impose  overwhelming  workloads  and  schedul¬ 
ing  constraints  resulting  in  too  few  quarters  for  study  of  too  much  material,  has 
virtually  precluded  any  reasonable  progress  toward  the  worthy  goal  of  creating  a 
meaningful  learning  environment  at  AFIT. 

On  the  other  hand,  as  results  of  this  thesis  reveal,  many  innovative  and 
creative  changes  can  be  made  in  the  commonplaces  of  the  Teaching  and  Learning, 
and  most  importantly,  to  the  Curriculum.  If  any  one  of  these  innovations  succeed, 
the  chances  for  a  more  meaningful  involvement  of  students  with  statistics,  in 
particular,  and  with  the  education  process,  in  general,  will  rise  drastically. 

Therefore,  while  the  goal  of  this  thesis  was  very  ambitious,  the  limitations 
imposed  on  this  particular  attempt  at  producing  a  more  meaningful  learning  en¬ 
vironment  were  severe.  Forever  optimistic  that  a  small  change  can  sometimes 
initiate  a  large-scale  transformation,  the  thesis  objectives  were  diligently  pursed. 
And,  as  will  be  be  reported  in  later  chapters,  while  it  appears  the  major  thrust  of 
the  statistical  program  is  consistent,  with  the  needs  of  the  field,  a  great  deal  more 
effort  must  be  given  to  designing  a  curriculum  that  fully  acknowledges  the  con¬ 
ceptual  maturity  of  each  student  and  which  makes  a  supreme  effort  at  addressing 
specific  examples  of  applications  that  demonstrate  the  utility  of  applying  statistics 
to  the  student.  Only  with  such  salesmanship  can  powerful  successes  within  the 
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commonplaces  of  Teaching  and  Learning  have  any  hope  of  perturbing  Governance 
to  a  new  center  of  gravity. 

With  Johnson’s  Model  and  Ausubel’s  Learning  Theory  in  hand,  the  research 
effort  proceeded  to  develop  a  survey  that  was  ultimately  submitted  to  the  field  of 
27XX  acquisition  managers  for  completion.  Specific  questions  posed  by  the  survey 
instrument  and  the  data  analysis  used  to  evaluate  the  questionnaire  are  outlined 
in  Chapter  III.  The  survey’s  findings  are  fully  documented  by  Chapter  IV. 


III.  Methodology 


Introduction 

As  discussed  in  Chapter  II,  the  establishment  of  a  meaningful  learning  envi¬ 
ronment  requires  decisions  be  made  concerning  what  is  to  be  taught  and  how  it  is 
to  be  taught.  The  methodology  presented  in  this  chapter  was  developed  to  discover 
the  conceptual  base  potential  students  can  be  expected  to  bring  to  AFIT  and  to 
identify  the  statistical  concepts  they  should  be  taught  in  statistical  courses  they 
take  at  AFIT.  The  chapter  itself  is  divided  into  four  major  sections;  survey  sample 
population  identification,  survey  instrument  development,  survey  administration, 
and  statistical  analysis  of  survey  data. 

Survey  Sample  Population  Identification 

The  population  of  interest  included  military  acquisition  systems  managers 
with  the  27XX  AFSC.  In  order  to  meet  the  research  objectives  and  answer  the 
research  questions,  it  was  determined  that  this  population  should  be  divided  into 
four  groups  as  follows:  AFSCs  2716  and  2724  with  and  without  an  AFIT  Systems 
Management  Masters  Degree.  This  is  illustrated  in  Figure  3.  The  sample  popula¬ 
tion  was  selected  using  a  modified  four-way  stratified  sampling  model.  The  sam¬ 
pling  of  acquisition  managers  (AFSCs  271X  and  272X)  without  an  AFIT  masters 
degree  was  accomplished  on  a  proportional  basis,  while  the  sample  of  acquisition 
managers  (AFSCs  271X  and  272X)  with  an  AFIT  masters  degree  involved  an  at¬ 
tempt  at  taking  a  full  census.  The  survey  collected  data  on  job  tasks  performed 
and  statistical  technologies  used  by  each  strata  of  interest. 

Personnel  were  identified  by  strata  from  data  contained  in  the  ATLAS  data 
base  maintained  by  the  Air  Force  Manpower  and  Personnel  Center  (AFMPC).  To 
be  selected  respondents  had  to  be  stationed  in  the  CONUS  area  and  hold  a  duty 
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AFSCs  of  2711,  2716,  2721,  or  2724.  The  distribution  of  ranks  and  number  of 
personnel  identified  by  ATLAS  (16)  are  shown  in  Table  1  and  the  distribution  of 
these  personnel  among  the  major  commands  identified  from  Officer  Authorizations 
Listing  (18)  is  shown  in  Table  2. 


V, 

V, 

v: 


Table  1:  Rank  Distribution  of  Survey  Population 


J  ■  j 

<* 

Total 

AFSC 

AFSC 

AFSC 

AFSC 

»* 

27xx 

27lx 

271x 

272x 

272x 

5* 

AFSC 

w/o 

with 

w/o 

with 

AED 

AED 

AED 

AED 

2nd  Lt 

360 

0 

0 

360 

0 

1st  Lt 

459 

0 

0 

455 

4 

Captain 

662 

65 

10 

507 

80 

i 

Major 

441 

333 

43 

57 

8 

Lt  Col 

475 

410 

61 

2 

2 

9 

Col 

112 

105 

7 

0 

0 

J 

2509 

913 

121 

1381 

94 

Based  on  sampling  statistics,  a  representative  sample  of  each  population  was 
obtained.  The  following  formula  was  used  for  determining  the  maximum  sample 
size  from  a  given  finite  population. 

Njz^xpjl-p) 


n  = 


(N-l)x(<P)  +  (z*)xp(l-p) 


(1) 
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Table  2:  MAJCOM  and  SOA  Distribution  of  Population' 


Organization 

AFSC 

271x 

AFSC 

272x 

Total 

27xx 

AFSC 

AFSC 

914 

1067 

1981 

AFCC 

8 

5 

13 

SAC 

7 

0 

7 

TAC 

3 

11 

14 

MAC 

7 

3 

10 

AFLC 

25 

39 

64 

SPC 

16 

16 

32 

Other 

55 

35 

90 

Air  Staff 

83 

11 

94 

HQ  AF 

126 

120 

246 

SAF 

4 

0 

4 

1248 

1307 

2555 

’  —  NOTE:  From  Officer  Authorizations  Listing ,  as  of  EOM  APR  8  (1986). 


where  n: 

N: 

P‘ 

d: 

z : 


sample  size 
population  size 

maximum  sample  size  factor  (0.50) 

desired  tolerance  (0.10) 

factor  of  assurance  for  confidence  level  (0.90) 


Using  Formula  1  the  return  sizes  shown  in  Table  3  were  required  to  achieve  a 
90  percent  confidence  level  and  ±10  percent  confidence  interval  (21).  Based  on 
an  expected  return  rate  of  60  percent,  the  calculated  sample  size  was  adjusted 
to  account  for  expected  non-returned  surveys.  This  new  sample  size  is  shown  in 
Table  4.  More  specific  sample  and  return  information  is  displayed  in  Table  5. 


Survey  Development 

The  data  collection  instrument  for  this  survey  was  27XX  Systems  Manager 
Survey,  SCN  87-77,  dated  8  June  1987.  The  survey  developed  to  answer  the  re- 
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Table  3:  Rank  Distribution  of  Survey  Sample  for  90  percent  Confidence  Level 


Total 

27xx 

AFSC 

AFSC 

271x 

w/o 

AED 

AFSC 

271x 

with 

AED 

AFSC 

272x 

w/o 

AED 

AFSC 

272x 

with 

AED 

2nd  Lt 

59 

0 

0 

59 

0 

1st  Lt 

78 

0 

0 

75 

3 

Captain 

165 

15 

7 

83 

60 

Major 

121 

76 

30 

9 

6 

Lt  Col 

137 

94 

42 

0 

1 

Col 

29 

24 

5 

0 

0 

589 

209 

84 

226 

70 

search  questions  contained  five  sections  and  can  be  found  in  Appendix  A.  The  first, 
a  background  section,  was  designed  to  gather  demographic  information  about  the 
respondents.  The  second  section  contains  a  list  of  61  task  functions  performed  by 
personnel  in  the  27XX  utilization  field,  these  job  task  functions  were  designed  to 
gather  information  about  the  level  of  involvement  with  decisions  and  actions  con¬ 
cerning  the  respondents.  The  third  section  is  composed  of  a  list  of  20  Professional 
Continuing  Education  (PCE)  courses  which  could  have  been  taken  by  entering 
AFIT  Students.  The  fourth  section  contains  a  list  of  10  academic  courses,  all  of 
which  are  related  to  statistics  and  are  available  at  all  levels  of  education.  These 
PCE  and  academic  courses  were  selected  to  gather  information  about  the  level  of 
completion  and  concept  utilization  relative  to  the  respondent.  The  last  section 
was  for  comments. 


Demographics.  In  this  part  of  the  survey,  the  respondent  was  requested  to 
provide  information  concerning  his/her  military  and  academic  background  and 
current  job.  These  items  were  rank,  duty  AFSC,  highest  academic  degree,  in 
what  field(s)  were  their  bachelors  (BS)  in,  if  they  had  an  AFIT  Masters  of  Science 
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Table  4:  Rank  Distribution  of  Survey  Sample 


Total 

27xx 

AFSC 

AFSC 

271x 

w/o 

AED 

AFSC 

271x 

with 

AED 

AFSC 

272x 

w/o 

AED 

AFSC 

272x 

with 

AED 

2nd  Lt 

98 

0 

0 

98 

0 

1st  Lt 

128 

0 

0 

124 

4 

Captain 

254 

25 

10 

139 

80 

Major 

194 

127 

43 

16 

8 

Lt  Col 

221 

156 

61 

2 

2 

Col 

54 

47 

7 

0 

0 

942 

355 

114 

379 

94 

(MS),  what  major  command  they  were  assigned,  what  their  current  job  was,  how 
many  people  did  they  supervise,  and  how  much  experience  as  a  27xx  did  they 
have. 


Survey  Task  Function.  In  the  task  function  section  of  the  survey,  respon¬ 
dents  were  asked  to  rate  each  task  on  a  5-point  scale  indicating  the  relative  amount 
of  decision  making  or  action  taken  on  that  task.  The  rating  scale  ranged  from  one 
(no  role)  to  five  (sole  role),  with  a  rating  of  three  representing  a  moderate  role  in 
performing  a  task. 

Factors.  The  job  function  areas  of  interest  are  listed  in  Table  6.  These 
eleven  areas  are  described  in  Air  Force  Regulation,  Officer  Air  Force  Specialty 
(AFR  36-1),  for  Acqisition  Management  Officer  (AFSC  2716)  and  Acquisition 
Project  Officer  (AFSC  2724).  These  areas  are  as  follows;  overall  program  man¬ 
agement,  program  office  management,  plans  and  manages  acquisition  programs, 
manages  personnel  subsystems  function,  performs  data  management,  assists  in 
configuration  management,  assists  in  program  control  performs  test  and  deploy- 
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Table  5:  Survey  Sample  Distribution 


Number  Percent 


Total  Personnel  Assigned  27xx  Duty  AFSC* 

2509 

Total  Eligible  Personnel  Surveyed 

942 

Percent  of  Assigned  Personnel  Surveyed 

37.5 

Total  Surveys  Returned 

556 

Percent  of  Returned  Surveys 

59.0 

Total  Returned  Undelieverable  Surveys 

28 

Percent  Undelieverable  Surveys 

3.0 

Total  Removed  From  Returned  Surveys*’ 

13 

Percent  Removed  From  Returned  Surveys 

1.4 

Total  Usable  Surveys 

515 

Percent  Usable  Surveys 

54.6 

•  -  As  of  12  May  1987,  ATLAS  STAT  SUMMARY  INQUIRY  16575. 

**  -  Excludes  surveys  from  personnel  with  less  than  6  months  experance  on  the 
job  and  eliminations  based  on  response  errors. 


ment  operations,  provides  acquisition  program  integrated  logistics  support,  staff 
functions,  and  acquisition  program  support  (17:A10-31-A10-34). 


Functions.  These  eleven  areas  can  be  reduced  to  six  task  functions; 
cost,  budget,  schedule,  technical  performance,  logistics,  and  administration.  The 
relationship  of  the  job  factors  with  the  task  functions  is  shown  in  Table  7.  The 
question  grouping  being  used  for  the  reduction  of  the  job  factors  to  the  six  function 
areas  can  be  seen  in  Table  8. 

Variables.  The  questions  selected  for  the  job  task  function  section  are 
the  variables.  These  questions  were  selected  and  designed  to  be  a  cross  section  of 
the  duties  performed  by  the  personnel  making  up  the  27xx  AFSC.  The  types  of 
duties  included  were  related  to  the  eleven  areas  shown  in  Table  6. 
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Table  6:  Acquisition  Management  Job  Factor  Areas 


Number 

AFSC 

Job  Factor 

1 

Overall  Program  Management 

2 

Program  Office  Management 

3 

2716/2724 

Plans  and  Manages  Acquisition  Programs 

4 

2716/2724 

Manages  Personnel  Subsystems  Function 

5 

2716/2724 

Performs  data  management 

6 

2716/2724 

Assists  in  Configuration  Management 

7 

2716/2724 

Assists  in  Program  Control 

8 

2716/2724 

Performs  Test  and  Deployment  Operations 

9 

2716/2724 

Provides  Acquisition  Program  Integrated 

Logistics  Support 

10 

2716 

Staff  Functions 

11 

2716 

Acquisition  Program  Support 

Professional  Continuing  Education  (PCE)  Courses.  In  the  third  section  of 
survey,  respondents  were  asked  to  answer  “if  they  had  completed  a  particular  PCE 
course,”  and  if  the  response  was  “YES,”  to  answer  the  next  part.  This  part  of 
the  question  asked  “if  they  had  found  use  for  the  course  material  in  their  current 
duties,”  and  if  so,  to  mark  the  response  which  was  correct  in  their  case.  The 
purpose  of  these  questions  was  to  identify  which  courses  an  entering  GSM  student 
can  be  expected  to  have  taken  before  entering  AFIT  and  whether  or  not  he  has 
used  the  concepts  covered  by  such  courses.  Courses  for  which  a  double  positive 
response  occurred  were  used  to  identify  the  applicability  of  PCE  courses  entering 
GSM  students  had  previously  taken.  This  information  was  also  used  to  help 
construct  the  expected  PCE  course  background  of  these  students.  A  number  of 
relevant  courses  were  left  out,  because  they  are  not  normally  available  to  company 
grade  officers  (i.e.,  Defense  Systems  Management  College’s  Program  Management 
Course). 
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Table  7:  Acquisition  Management  Job  Factor  Areas  verses  Task  Function 


Job 

Factor 

Cost 

Budget 

Schedule 

Technical 

Logistics 

Administration 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

5 

X 

X 

X 

6 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

10 

X 

X 

11 

X 

X 

X 

X 

Academic  Statistical  Courses.  In  the  fourth  section  of  the  survey  respon¬ 
dents  were  asked  to  complete  the  same  type  of  questions  as  in  the  third  section. 
These  questions  were  designed  to  identify  which  statistical  courses  an  entering 
GSM  student  could  be  expected  to  have  taken  before  attending  AFIT  and  whether 
or  not  he  had  used  the  concepts  covered  by  the  course.  As  was  done  with  the  PCE 
courses,  academic  courses  for  which  a  double  positive  response  occurred  were  used 
to  identify  the  applicability  of  statistical  courses  entering  GSM  students  had  pre¬ 
viously  taken.  This  information  was  also  used  to  help  construct  the  expected 
statistical  course  background  of  these  students.  The  statistical  course  topics  in¬ 
cluded  in  the  survey  were  selected  with  the  availability  of  the  courses  to  field 
personnel  in  mind. 

Comments.  The  last  section  of  the  survey  was  for  comments.  This  is  where 
the  respondent  could  address  any  topic  he  felt  needed  to  be  addressed  either 
because  of  an  omission  of  an  important  item  in  the  survey  or  because  he/she  just 
wanted  to  make  a  personal  observation. 
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Table  8:  Survey  Items  Categorized  by  Task  Functions 


Job  Task  Function 

Survey  Item  Number 

Cost 

10,  18,  32,  34,  36, 

38,  41,  49,  55,  60, 

63,  65 

Budget 

17,  20,  22,  24,  26, 

30,  40 

Schedule 

25,  29,  45,  57,  62 

Technical  Performance 

11,  16,  35,  39,  42, 

43,  48,  54,  58,  64, 

66,  68 

Logistics 

13,  14,  21,  28,  31, 

51,  59,  67,  69 

Administration 

12,  15,  19,  23,  27, 

33,  37,  44,  46,  47, 

50,  52,  53,  56,  61, 

70 

Survey  Administration 

From  June  1987  through  August  1987,  27XX  Systems  Manager  Surveys  were 
administered  to  a  randomly  selected  group  of  personnel  in  the  field  of  system  ac¬ 
quisition  management  (AFSC  27XX).  The  surveys  were  distributed  to  the  subjects 
through  the  mail  using  the  BITS  system  whenever  possible.  The  subjects  were 
sent  the  survey  by  name  through  their  organization  of  record.  The  package  the 
subjects  received  contained  the  survey  and  a  return  envelope.  The  survey  con¬ 
tained  instructions  on  how  to  fill  out  the  survey  and  how  to  return  the  survey. 
The  surveys  were  returned  by  mail  to  AFIT/LSG  by  the  individual  subject  in  the 
provided  envelopes.  When  the  survey  arrived  at  AFIT/LSG  there  was  no  way  to 
identify  the  returned  survey  with  any  of  the  individuals  sent  surveys.  This  way 


the  individual  was  granted  anonymity.  The  surveys  were  then  coded  on  to  optical 
scan  forms  for  data  processing. 

Statistical  Analysis  of  Survey  Data 

Surveys  returned  from  the  field  were  entered  into  the  AFIT  computer  at 
Wright- Patterson  AFB  by  optical  scanning.  The  Statistical  Analysis  System 
(SAS)  was  then  used  to  analyze  the  data.  The  SAS  program  is  capable  of  produc¬ 
ing  a  variety  of  computer  printouts  based  on  survey  respondent  ratings  on  tasks, 
PCE  courses,  and  academic  courses,  and  background  information. 

Reliability  Check.  Cronback’s  alpha  was  calculated  in  order  to  make  a  task 
function  question  reliability  check.  Questions  10  through  70  were  run  through  the 
reliability  check.  It  was  decided  that  an  alpha  value  greater  than  or  equal  to  0.8 
would  consistute  an  adequate  reliability  coefficient. 

Format  Used  for  Developing  the  Research  Questions.  The  following  format  is 
used  in  developing  all  of  the  research  questions  methodology.  The  topics  discussed 
in  each  question  area  are  the  population  of  interest,  research  hypothesis,  statistical 
test  selection  or  descriptive  procedures,  test  statistic  if  needed,  assumptions,  focus 
if  the  null  is  rejected,  and  limitations.  In  the  following  sections  on  the  research 
questions  the  specific  methods  of  analysis  will  be  discussed. 

Research  Questions  1  and  2.  The  first  two  research  question  that  needed  to 
be  answered  were, 

Research  Question  1:  What  coursework  and,  more  specifically,  what  exposure  to 
statistical  concepts  via  academic  and/or  Professional  Continuing  Education  study 
have  potential  GSM  students  received?  and 
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Research  Question  2:  What  statistical  concepts  have  found  application  in  the  en¬ 
vironment  of  potential  GSM  students  ? 

Population  of  Interest.  The  population  of  interest  in  both  cases  was 
the  27xx  without  an  AFIT  masters  degree  and  holding  a  grade  of  major  or  below. 
This  subset  of  the  survey  data  was  selected  because  it  is  representative  of  the 
population  of  the  potential  students  for  the  GSM  program. 

Research  Hypothesis  The  research  hypothesis  for  both  of  these  ques¬ 
tions  was  that  there  is  an  association  between  whether  a  person  takes  a  course 
and  whether  he  is  able  to  use  that  course. 

Statistical  Test  Selection.  The  statistical  test  selected  was  a  Test  for 
Independence.  An  example  of  the  table  used  is  shown  in  Figure  4.  The  research 
hypothesis  was  converted  to  the  following  set  of  statistical  hypotheses. 

Ha  :  Pt]  =  Pi  ■  Pj\  for  i  =  YES,  NO  and  j  =  YES,  NO  (2) 

Ha:Ha  is  not  true  (3) 

Where  :  is  the  probability  for  that  cell 

pp.  is  the  probability  for  the  category  of  completed  course 

Pji  is  the  probability  for  the  category  of  used  course  on  the  job 

Test  Statistic.  \2  was  used  as  the  test  statistic  for  this  test.  Its  use 
requires  that  probability  values  for  each  cell  must  be  found.  The  values  of  p,. 
and  p.j  are  not  readily  available,  therefore  must  be  estimated.  Devore  derives  the 
“estimated  expected  cell  count”  (5:550)  and  presents  the  following  proposition: 


QXXA (Completed  course) 


QXXB(Used  course) 


FREQUENCY 

PERCEHT  YES  I  NO  I  TOTAL 


YES 

— +- 
1 

1 

NN 

yy-yy 

■+- 

1 

1 

NN 

yy-yy 

■+ 

l 

l 

MM 

kk.kk 

NO 

1 

NN 

yy-yy 

1 

NN 

yy-yy 

l 

MM 

kk.kk 

TOTAL 

pp 

qq-qq 

pp 

qq-qq 

XX 
sss . ss 

FREQUENCY  MISSING  =  qqq 


STATISTICS  FOR  TABLE  OF  qXXA  BY  QXXB 

STATISTIC  DF  VALUE  PROB 


CHI-SQUARE  N  x.xxx  z.zzz 


Figure  4:  Two-Way  Contingency  Table  Example 


When  the  null  hypothesis  of  independance  of  factors  is  true  and  EtJ  >  5 
for  all  i,  j,  the  statistic 


2  ^Wa-Eii)2 


X  -1*1* 
«=1 ;=1 


E, 


V 


has  approximately  a  chi-squaxed  distribution  with  (/— 1)(7  — 1)  degrees 
of  freedom  (u).  A  test  for  H0  versus  Ha  which  has  approximate  level 
a  consists  of  rejecting  H0  if  \2  >  xl  (/— i)(V— i)  (5:551). 


An  alpha  of  0.05  was  used  to  establish  the  critical  value  for  the  test  statistic. 
According  to  Devore  the  formula  for  degrees  of  freedom  (u)  is, 


(4) 


In  this  situation,  u  is 


Where  /:  is  2 
J :  is  2 

Using  this  information  and  a  statistical  table  for  x2>  the  critical  value  of  X2,(/_i)(j-i) 
was  found  to  be  3.843. 

Decision  Rule.  If  x2  >  3.843  reject  H0  in  favor  of  Ha. 

Assumptions.  The  statistical  and  practical  assumptions  for  this  test 
are  discussed  below. 

Statistical.  The  statistical  assumptions  were  as  follows; 

1.  An  independent  random  sample  has  been  taken. 

2.  The  expected  frequency  of  each  cell  is  at  least  five  (5). 

3.  Each  respondent  belongs  to  one  and  only  one  of  the  I  categories  for  completed 
course  and  one  and  only  one  of  the  J  categories  for  used  course  on  the  job. 

Practical.  The  practical  assumption  was  that  the  individuals 
surveyed  understand  at  least  enough  about  acquisition  management  to  make  state¬ 
ments  concerning  course  application  to  their  current  duties. 

Focus,  if  the  Null  is  Rejected.  Only  if  the  Yes/Yes  response  in  the  table 
was  found  to  exceed  70  percent  of  the  total  responses,  was  the  course  considered 
important  enough  to  be  included  in  the  set  of  courses  that  would  be  used  to  answer 
research  questions  1,  2,  and  4. 

Limitations.  The  data  used  to  answer  research  questions  1  and  2  was 
garnered  under  the  following  constraints: 


1.  The  data  could  not  be  used  to  predict  an  estimate  statistical  concept  struc¬ 
ture  of  any  particular  entering  GSM  student,  but  instead  would  be  an  indica¬ 
tor  of  the  typical  cognitive  structure  brought  to  AFIT  by  entering  acquisition 
management  GSM  students. 

2.  Due  to  the  limitation  of  space  and  respondents  time  the  number  of  courses 
was  restricted  to  twenty  PCE  and  ten  statistical  courses. 

Research  Question  S.  The  research  question  was,  What  particular  job  func¬ 
tions  are  currently  being  accomplished  by  acquisition  managers? 

Population  of  Interest.  The  population  of  interest  in  this  case  was  the 
entire  sampled  27xx  population. 

Research  Hypothesis.  The  research  hypothesis  for  this  question  was 
that  the  functions  could  be  ranked  based  on  their  mean  task  function  scores. 

Statistical  Test  Selected.  The  statistical  test  selected  was  an  analysis 
of  variance  (ANOVA).  The  research  hypothesis  was  converted  to  the  following  set 
of  statistical  hypotheses: 

H0  •  fJ-coit  P budget  ~  P schedule  =  Ptechn  =  Plog  =  Pa dmin  (6) 

Ha  •  Pcoti  ^  Pbudget  ^  P schedule  5^  Ptechn  ^  Plog  Padmin  (7) 


ANALYSIS  OF  VARIANCE  PROCEDURE 
DEPENDENT  VARIABLE:  XXXX 

SOURCE  DF  SUM  OF  SQUARES  MEAN  SQUARE 

MODEL  7-1  $TLiXl-h& 

error  i(j-i)  eLi  e/=i(*u  -  *-)2  >3 

CORRECTED  TOTAL  IJ  -  1  ELx  E/=i  -  ^X.2 


*T7  'X}-bx* 

12*im&irL 


MODEL  F  = 


.  ~  . 
E,.i  ,J 

- rtTrrj - 


PR  >  F  =  x.x 


Figure  5:  Analysis  of  Variance  Illustration 


where  nco,t: 

is  the  sample  mean  of  the  cost  task 
function  area. 

Hbud- 

is  the  sample  mean  of  the  budget  task 
function  area. 

Hsch- 

is  the  sample  mean  of  the  schedule  task 
function  area. 

f^techn- 

is  the  sample  mean  of  the  technical 
performance  function  area. 

f*log- 

is  the  sample  mean  of  the  logistics  task 
function  area. 

Hadmin'- 

is  the  sample  mean  of  the  administration 
task  function  area. 

Ha  is  to  be  read  as  at  least  one  group  mean  is  different  from  another.  The 
null  hypothesis  is  that  the  means  sire  the  same,  while  the  alternate  hypothesis  is 
th  ,t  at  least  two  means  are  different.  An  example  of  the  ANOVA  output  used  is 
shown  in  Figure  5. 
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Test  Statistic.  The  F  test  was  used  as  the  test  statistic  for  this  test. 
To  use  the  F  test,  two  variances  must  be  estimated.  These  variances  axe  denoted 
by  for  the  between-sample  estimator  and  a w  for  the  within-sample  estimator. 
The  formula  for  the  is 


.2  T„2  ZL  £/=1(*.-x.)2 

aB  ~  J  ~  I  -  l - 


(8) 


and  the  formula  for  &  w  is 


Ef=1(^-X,.)2 

7(7-1) 


(9) 


The  F  test  then  used  the  formulas  above  in  the  form  shown  below, 


F.i i 

*  *9 


(10) 


In  order  to  use  the  formula  for  F  to  find  the  critical  value  of  F  for  this  test 
the  value  of  a,  the  numerator  degrees  of  freedom  (i/i),  and  the  denominator  degree 
of  freedom  (i/2)  must  be  identified  and  found.  A  value  of  0.05  was  selected  for  a. 
The  value  of  vx  is  found  using  the  formula, 


=  7-1  =  6— 1  =  5 
The  value  of  i/2  is  found  using  the  following  formula, 


(ID 


i/2  =  7(7  -  1)  =  6(515  -  1)  =  3084  (12) 

With  the  above  values  and  an  F  table  (5:624-625)  for  F,  the  critical  value  of 
Fa,l-\,l(j-i)  was  found  to  be  2.21  (5:343-353). 


Decision  Rule.  The  decision  rule  for  the  F  test  is  to  reject  H0  in  favor 
of  Ha  if  F  >  2.21. 


Assumptions.  The  statistical  and  practical  assumptions  for  this  test 
are  discussed  below. 
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Statistical.  The  statistical  assumptions  were  as  follows; 


1.  “The  variables  Xij(i  =,  . . . ,  I  and  j  =  1,  . . . ,  J)  axe  independent  of  one 
another  with  E(X{j)  =  /*,,  Var(Xij)  =  <t2”  (5:347). 

2.  Each  Xij  is  a  normally  distributed  variable,  if  all  I  populations  were  normal 
populations  (5:348). 

Practical.  The  practical  assumption  was  that  the  individuals 
surveyed  understand  at  least  enough  about  acquisition  management  to  make  state¬ 
ments  concerning  course  application  to  their  current  duties. 

Focus,  if  the  Null  is  Rejected.  If  Ha  was  rejected,  the  Tukey  procedure 
was  to  be  used  to  establish  the  ranking  of  the  functions  based  on  the  mean  task 
function  scores. 

Limitations.  The  information  gathered  to  answer  research  question  3 
is  unsuitable  for  developing  an  estimation  of  the  job  task  structure  in  any  given 
program/project,  but  is  instead  is  an  indicator  of  the  relative  job  task  structure 
used  by  acquisition  managers. 

Research  Question  4.  The  fourth  research  question  was,  Given  a  particular 
job  function  what  statistical  technologies  are  being  used  to  help  solve  management 
problems  associated  with  that  job  function? 

Population  of  Interest.  The  population  of  interest  in  this  case  was  the 
entire  27xx  population. 
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Figure  6:  Course  Job  Factor/Task  Correlation  Description  Illustration 

Motivation  for  Research  Question  4 ■  The  motivation  for  this  question 
was  to  discover  if  a  relationship  between  statistical  courses/statistical  concepts 
and  specific  functions  could  be  inferred. 

Descriptive  Procedure.  The  end  result  of  the  procedure  established  to 
answer  the  research  question  4  was  a  table  of  counts  in  the  form  of  Figure  6.  Each 
respondents  ranking  of  the  six  task  functions  was  correlated  with  the  populations 
ranking  of  the  functions  and  then  used  to  categorize  each  individual  into  a  task 
correlation  group  within  one  of  three  activity  groups.  Then  for  each  respondent 
an  accounting  was  made  of  any  course  containing  a  double  positive  response  if  the 
course  had  been  identified  during  the  resolution  of  research  questions  1  and  2  as 
containing  a  significant  number  of  double  positive  responses.  A  large  number  of 
individuals  in  a  high  positive  correlation  task  function  group  and  course  cell  would 
be  taken  as  sufficient  evidence  to  support  an  inference  that  concepts  taught  in  the 
course  were  being  widely  applied  by  27xxs  in  that  activity  group. 
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Summary 

In  this  chapter,  the  methodology  has  been  developed  that  was  used  to  dis¬ 
cover  the  conceptual  base  potential  students  can  be  expected  to  bring  to  AFIT 
and  to  identify  the  statistical  concepts  that  should  be  taught  in  statistical  courses 
they  will  take  at  AFIT.  Chapter  IV  will  present  the  results  of  application  of  the 
methodology  proposed  by  this  chapter. 
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IV.  Analysis  of  Survey  Results 


Introduction 

This  chapter  analyzes  the  findings  and  results  of  the  survey  conducted  by 
this  thesis  effort.  It  is  divided  into  two  major  sections: 

1.  Presentation  and  analysis  of  demographic  data,  and 

2.  Presentation  and  analysis  of  answers  provided  by  the  survey  to  each  research 

question. 

Presentation  and  Analysis  of  Demographic  Data 

The  surveys  were  mailed  to  942  personnel  assigned  the  duty  AFSC  of  271x 
or  272x.  Overall  556  surveys  were  returned.  Of  the  556  surveys  received,  28  were 
returned  unopened  and  13  were  unusable  (due  to  personnel  not  having  more  than 
six  months  experience  as  a  27xx  and  response  errors).  This  left  515  surveys  to 
build  the  research  data  base  with  (54.3  percent  response  rate). 

Findings  and  Analysis.  The  respondent  supplied  the  following  information 
concerning  his  military  and  academic  background  and  current  job;  rank,  duty 
AFSC,  highest  academic  degree,  in  what  field(s)  were  their  bachelors  (BS)  in, 
if  they  had  an  AFIT  Masters  of  Science  (MS),  what  major  command  they  were 
assigned,  what  their  current  job  was,  how  many  people  did  they  supervise,  and 
how  much  experience  as  a  27xx  did  they  have. 

The  “typical”  acquisition  manager  can  be  described  by  using  the  modal  cat¬ 
egories  of  the  demographic  questions,  shown  in  Table  9.  He  is  a  captain  with 
a  2724  AFSC.  He  possesses  a  bachelors  degree  in  engineering/ technology  and  a 
non- AFIT  masters  degree.  Assigned  to  the  Air  Force  Systems  Command  (AFSC) 
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he  performs  the  duties  of  a  program /project  manager  that  require  supervision  of 
himself  or  one  other  person.  He  typically  has  two  to  four  years  of  acquisition  expe¬ 
rience.  The  demographic  information  from  the  data  base  is  displayed  in  Appendix 
C  (in  Figures  7  through  16). 


Table  9:  Summary  of  Demographic  Data 


Question  #  and  Topic 

Modal  Response 

Percent  Response 

Q1  -  Rank 

Captain 

Q2  -  AFSC 

2724 

Q3  -  BS/MS/PhD 

MS 

62.1 

Q4  -  BS  Major 

Engineering/Technical 

59.8 

Q5  -  AFIT  Graduate  or  not 

Non-AFIT  Graduate 

69.1 

Q6  -  Major  Command 

AFSC 

79.0 

Q7  -  Duty  Title 

Program/Project  Manager 

46.6 

Q8  -  Supervisor 

0-1  people 

58.4 

Q9  -  Acquisition  Experience 

2  years  j  4  years 

31.8 

Cronback’s  alpha  reliability  coefficient  was  used  to  assess  the  reliability  of 
the  survey  (shown  in  Appendix  A).  Coefficient  alpha  and  the  standardized  alpha 
were  both  greater  than  0.97,  hence  well  above  the  minimum  value  established  for 
declaring  attainment  of  sufficient  reliability  in  chapter  III.  The  coefficient  of  alpha 
for  each  question  was  found  to  be  greater  than  0.97.  This  information  is  also 
available  in  Appendix  C  in  Table  17  through  18. 

Presentation  and  Analysis  of  Answers  Provided  by  the  Survey 

Research  Question  1  and  2.  The  two  research  questions  were, 

1.  What  coursework  and,  more  specifically,  what  exposure  to  statistical  concepts 
via  academic  and/or  Professional  Continuing  Education  study  have  potential 
GSM  students  received ? 
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2.  What  statistical  concepts  have  found  application  in  the  environment  of  po¬ 
tential  GSM  students? 

Findings.  The  respondents  used  in  this  part  supplied  information  con¬ 
cerning  PCE  course  exposure  and  utilization.  The  data  may  be  found  in  the  sum¬ 
mary  table  displayed  in  Appendix  C  (in  Table  12).  Courses  listed  below  were  found 
to  meet  the  criteria  of  exceeding  the  statistical  test  critical  value  and  of  exceeding 
the  level  of  70  percent  for  the  category  taken  and  used  as  defined  in  chapter  III. 
According  to  the  data  the  “typical”  potential  student  has  taken  the  following  PCE 
courses  that  he  has  found  use  for  on  the  job;  SAS  001/SYS  100/SYS  123,  SYS 
028,  SYS  200/SYS  223,  SYS  229,  and  DSMC  CPM. 

The  following  descriptions  of  the  above  courses  axe  provided  to  show  the 
concepts/content  of  the  courses. 

SAS  001/SYS  100/SYS  123.  The  courses  SAS  001,  Introduc¬ 
tion  to  systems  Command  Acquisition  Management;  SYS  100,  Introduction  to 
Acquisition  Mangement;  and  SYS  123,  Fundamentals  of  Acquisition  Mangement; 
are  grouped  together  as  they  are  very  similar  courses.  These  courses  provide  the 
following,  “[c]urrent  concepts  of  acquisition  management  (DoDD  5000.1  and  AFR 
800-2)  and  problem  areas  in  the  acquisition  process”  (19:51). 

SYS  028.  This  course  SYS  028,  Introduction  to  Configuration 
Management  provides  the  following,  “basic  philosophy  and  practices  of  config¬ 
uration  Management  . . .  [relating  to]  the  key  areas  of  configuration  management 
(naunely  identification,  audits,  change  control,  and  status  accounting)  ...”  (19:30). 

SYS  200/SYS  223.  The  courses  SYS  200,  Acquisition  Planning 
and  Analysis  and  SYS  223,  System  Program  Management  provide  the  following, 


“content  in  three  management  functional  areas:  planning,  executing,  and  control¬ 
ling  . . .  such  as  generating  a  POM,  writing  a  SOW,  using  cost  estimating  resources, 
analyzing  contractor  performance  and  developing  planning  networks”  (19:69). 

SYS  229.  This  course  SYS  229,  Test  and  Evaluation  Manage¬ 
ment  provides  the  following,  . .  test  planning,  test  reporting,  and  special  issues 
. . .  insight  into  test  management  and  constraints,  and  difficulties  ...  in  any  T&E 
program”  (19:85). 

DSMC  CPM.  This  course  DSMC  CPM,  Defense  Systems  Man¬ 
agement  College;  Contractor  Performance  Measurement  provides  the  following, 
“instruction  in  analysis  techniques  [to]  enable  . . .  the  student  to  determine  current 
status,  forecast  performance  trends,  and  estimate  . . .  contract  cost  at  completion” 
(4:5-F-5). 

Respondents  also  supplied  the  information  concerning  academic  statistic 
course  exposure  and  utilization.  According  to  the  data  the  “typical”  potential 
student  appears  to  have  not  taken  and  used  any  of  the  statistics  courses  listed 
in  the  survey  by  the  same  criteria  used  above  for  acceptance  of  the  course  for 
consideration  as  defined  in  chapter  III.  This  data  may  be  found  in  the  summary 
table  displayed  in  Appendix  C  (in  Table  13). 

Analysis.  The  courses  identified  might  refer  to  statistics  theory  but 
they  do  not  directly  teach  statistical  concepts.  This  leads  to  am  inference  that 
the  potential  acquisition  manager  GSM  student  has  not  had  previous  course  work 
in  statistics  and  the  the  exposure  to  applied  statistics  lacked  the  rigor  of  formal 
statistics  courses.  In  spite  of  the  fact  that  several  written  comments  on  the  survey 
responses  indicated  a  need  for  statistical  education  and  application,  it  appears 


that  the  “typical”  potential  student  has  not  found  any  application  for  statistical 
courses  he  may  have  taken. 

A  further  inference  can  be  made  that  the  acquisition  managers  do  not  use 
statistical  techniques  directly.  On  the  other  hand,  while  the  27xxs  may  not  use 
or  perform  statistical  calculations  themselves,  they  do  have  occasions  to  use  or 
interpret  statistics  presented  by  others.  The  following  comment  taken  from  one  of 
the  surveys  returned  supports  this. 

. . .  More  important  . . .  than  understanding  the  technical  details  of  var¬ 
ious  statistical  methods,  the  greatest  value  of  statistical  courses  ...  to 
me  as  an  acquisition  manager  has  been  learning  to  question  input  pa¬ 
rameters,  assumptions  and  output  sensitivities!  (  See  Appendix  B) 

While  the  survey  indicates  statistics  is  not  viewed  as  having  great  relevance, 
written  comments  confirm  the  vitial  nature  of  statistical  education  to  the  practic¬ 
ing  acquisition  manager. 

Statistics  are  misapplied  so  often  that  everyone  should  know  how  statis¬ 
tics  works.  Specifically,  in  evaluating  weapon  system  performance,  it 
is  critical  to  recognize  the  proper  distribution  to  use,  sample  size,  and 
where  varying  Test  Conditions  invalidate  test  results.  Unfortunately, 
a  high  percentage  of  Air  Force  Testing  appears  to  not  '  ,  good  sam¬ 

ple.  Cost  of  assets  and  the  numerous  variables  needed  o  duplicate 
the  operational  environment  (which  create  the  need  for  large  sample 
sizes)  often  causes  the  evaluation  of  a  weapon  system’s  performance  to 
degenerate  into  subjective  feelings  and  guesses  rather  than  objective 
evaluation  based  on  an  adequate  sample  size  in  testing  (See  Appendix 
B,. 

Such  comments  together  with  the  survey  data  results  indicate  that  while  statistics 
is  presently  not  a  major  factor  in  performing  the  acquisition  process  to  the  27XX, 
it  should  be. 


Research  Question  3.  The  third  research  question  was,  What  particular  job 
functions  are  currently  being  accomplished  by  acquisition  managers ? 


Findings.  The  ANOVA  was  run  as  set  forth  in  the  methodology,  and 
the  results  can  be  found  in  Appendix  C  (in  Table  15).  The  F  value  was  found 
to  be  69.24,  which  exceeded  the  critical  value  of  2.21  determined  in  chapter  III. 
Therefore,  the  research  hypothesis  that  the  functions  could  be  ranked  based  on 
their  means  was  supported.  Through  the  use  of  the  Tukey  process,  it  was  found 
there  were  four  distinct  mean  groupings.  These  are  shown  in  Table  10.  Also, 
shown  in  the  table  are  the  rankings  of  the  functions  in  following  order  (based 
on  mean  score);  schedule,  administration,  budget,  technical  performance,  cost, 
and  logistics.  The  starred  lines  indicate  there  is  no  clear  distinction  between  the 
means  of  the  functions  in  that  grouping.  The  groups  involved  axe  logistics/cost 
and  budget/administration.  The  analysis  section  will  relate  these  findings  to  the 
research  question. 

Table  10:  Summary  of  ANOVA  and  Tukey  results  for  the  six  Task  Functions  Data 

Rank:  6  5  4  3  2  1 

Function:  Logistics  Cost  Technical  Budget  Administration  Schedule 

Means:  1.730  1.801  2.127  2.311  2.321  2.574 

*******************  ************************* 

Non-Quant  Quant  Quant  Quant  Non-Quant  Quant 


Analysis.  This  ranking  reflects  the  relative  level  of  decision  making 
and/or  action  talcing  done  by  these  managers.  The  judgemental  functions  (Non- 
Quant)  do  not  have  a  need  for  the  statistical  methods,  while  the  quantitative 
functions  (Quant)  do  use  analytical  mathematical  methods. 

Based  on  the  results,  shown  in  the  table,  scheduling  is  the  function  performed 
most  by  acquisition  managers,  followed  by  administration,  budget,  and  technical 


performance.  Thus  three  out  of  the  four  most  highly  ranked  functions  acquisition 
managers  perform  are  quantitative  in  nature.  This  is  a  curious  result  considering 
the  indication  provided  by  responses  to  research  questions  1  and  2  which  imply 
the  most  useful  courses  are  of  a  non-statistical/mathematical  nature.  The  lack  of 
statistical  sophistication  implied  by  these  two  findings  may  indicate  why  systems 
acquisition  has  experienced  problems  such  as  schedule  delays  and  cost  overruns. 

Research  Question  4-  The  fourth  research  question  was.  Given  a  particular 
job  function  what  statistical  technologies  are  being  used  to  help  3olve  management 
problems  associated  with  that  job  function? 

Findings.  The  descriptive  procedure  was  run  as  set  forth  in  the  method¬ 
ology.  The  results  of  this  procedure  can  be  found  in  Table  11.  As  shown  in  the 
table  there  were  no  respondents  in  the  negative  correlation  category.  While  the 
remaining  two  categories  received  the  rest  of  the  respondents  correlation  scores, 
the  largest  portion  went  to  the  unclear  correlation  category,  with  94.4  percent  of 
the  respondents  in  the  data  base.  This  left  the  positive  correlation  category  with 
5.6  percent  of  the  survey  populations.  The  only  category  with  large  numbers  of 
counts  in  them  were  SAS  001,  SYS  100,  and  SYS  200  and  these  counts  were  in 
the  unclear  range  of  the  correlation  categories.  This  would  indicate  that  concepts 
of  these  courses  get  a  great  deal  of  use. 

Analysis.  Based  on  these  cell  counts,  there  appears  to  be  no  way  to 
associate  the  statistical  concepts  to  a  particular  activity  job  group.  Indeed,  there 
are  simply  no  statistical  courses  represented  at  all. 

Chapter  V  will  present  the  conclusions  drawn  from  answers  to  these  four 
research  questions  and  make  recommendations  for  further  research. 
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Table  11:  Summary  of  Course  Activity/Correlation  Data 


Activity 

Group 

Task 

Function 

Correlation 

Brooks 

SAS 

001 

YES/ 

YES 

AFIT 

SYS 

028 

YES/ 

YES 

AFIT 

SYS 

100 

YES/ 

YES 

AFIT 

SYS 

123 

YES/ 

YES 

AFIT 

SYS 

200 

YES/ 

YES 

AFIT 

SYS 

223 

YES/ 

YES 

AFIT 

SYS 

229 

YES/ 

YES 

DSMC 

CPM 

YES/ 

YES 

Program/ 

Positive 

2 

0 

0 

0 

2 

0 

0 

0 

Project 

Unclear 

84 

25 

99 

14 

87 

25 

16 

19 

Manager 

Negative 

0 

0 

0 

0 

0 

0 

0 

0 

Functional 

Positive 

2 

2 

4 

1 

4 

0 

0 

0 

Manager 

Unclear 

49 

18 

54 

6 

33 

17 

14 

14 

Negative 

0 

0 

0 

0 

0 

0 

0 

0 

Others 

Positive 

2 

1 

urn 

0 

6 

3 

2 

3 

0-g.,  HQ 

Unclear 

16 

9 

ns 

11 

25 

20 

13 

13 

AF,  PEMs) 

Negative 

0 

0 

■ 

0 

0 

0 

0 

0 

All 

Positive 

6 

3 

8 

1 

KBI 

3 

2 

n 

Unclear 

149 

53 

198 

32 

KE9 

62 

44 

HH 

Negative 

0 

0 

0 

0 

■  —  ■  - 

0 

0 

0 

0 

Where  Positive: 

Unclear: 

Negative: 


(0.5)  <  (1.0) 
(-0.5)  <  (0.5) 
(-1-0)  <(-0.5) 
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V.  Conclusions  and  Recommendations 


Summary 

The  purpose  of  this  thesis  was  to  collect  basic  information  on  the  statistical 
concept  base  potential  GSM  students  might  bring  to  AFIT  and  to  determine  which 
statistical  concepts  are  used  by  acquisition  managers.  The  specific  application  of 
Ausubel’s  Learning  Theory  and  Johnson’s  Curriculum  Model  was  to  statistics, 
even  though  both  can  be  applied  to  other  disciplines.  The  Theory  and  the  Model 
were  discussed  in  Chapter  II. 

In  order  to  identify  the  statistical  concepts  entering  students  can  be  expected 
to  possess  and  in  order  to  facilitate  an  environment  for  meaningful  learning  this 
thesis  sought  to  meet  the  following  objectives: 

1.  To  collect  data  on  statistical  background  and  work  experience  of  acquisition 
managers,  so  an  inference  could  be  made  concerning  the  statistical  concept 
base  of  entering  GSM  students, 

2.  To  determine  the  job  functions  being  performed  by  acquisition  managers, 
so  that  problems  and  exercises  for  statistical  courses  can  be  developed  to 
represent  the  types  of  situations  GSM  students  can  be  expected  to  encounter 
upon  graduation  from  AFIT. 

In  developing  the  methodology,  in  Chapter  III,  to  meet  objective  1  the  fol¬ 
lowing  research  questions  were  addressed: 

1.  What  coursework  and,  more  specifically,  what  exposure  to  statistical  con¬ 
cepts  via  academic  and/or  Professional  Continuing  Education  study  have 
potential  GSM  students  received? 


2.  What  statistical  concepts  have  found  application  in  the  environment  of  po¬ 
tential  GSM  students? 

And  to  meet  objective  2  the  following  research  questions  were  addressed: 

3.  What  particular  job  functions  are  currently  being  accomplished  by  Acquisi¬ 
tion  Managers? 

4.  Given  a  particular  job  function,  what  statistical  technologies  are  being  used 
to  help  solve  management  problems  associated  with  that  job  function? 

The  surveys  were  mailed  to  942  personnel  assigned  the  duty  AFSC  of  271x 
or  272x.  Overall  556  surveys  were  returned.  Of  the  556  surveys  received,  28  were 
returned  unopened  and  13  were  unusable  (due  to  personnel  not  having  more  than 
six  months  experience  as  a  27xx  or  response  errors).  This  left  515  surveys  with 
which  to  build  the  research  data  base  with  (54.3  percent  response  rate).  These 
surveys  were  analyzed  and  the  results  shown  in  chapter  IV.  The  conclusions  drawn 
from  these  results  are  discussed  next. 

Conclusions 

Three  conclusions  were  reached  and  these  were, 

1.  While  there  are  indications  that  statistics  is  needed  by  acquisition  managers, 
the  current  survey  results  show  that  the  acquisition  managers  are  not  using 
any  of  the  identified  statistical  course/concepts  in  their  work. 

2.  While  the  top  ranked  task  functions  are  primarily  quantitative  in  nature, 
the  courses  most  used  are  qualitative  with  little  or  no  quantitative  nature. 

3.  WTiile  the  response  was  good  to  the  survey,  the  use  of  this  survey  to  try  and 
infer  the  concepts  entering  acquisition  management  GSM  students  would 
bring  to  AFIT  produced  unclear  results.  Any  attempt  to  infer  an  association 


among  the  statistical  concepts  with  the  activity  job  groups  will  most  likely 
require  personal  interviews  of  selected  field  personnel. 

Recommendations 

The  following  three  recommendations  for  future  research  are  proposed, 

1.  The  use  of  personal  interviews  of  acquisition  managers  should  be  used  to 
discover  the  real  need  of  acquisition  managers  for  various  statistical  disci¬ 
plines. 

2.  A  way  to  assess  the  cognitive  structure  of  the  entering  GSM  student  upon 
entry  to  AFIT  should  be  developed. 

3.  Serious  consideration  should  be  given  to  what  concepts  ought  to  be  taught 
and  to  new  ways  to  teach  such  concepts  so  the  general  lack  of  appreciation  of 
the  real  world  value  of  statistical  technology  in  the  acquisition  management 
field  can  be  overcome. 

The  fact  is  project  management  requires  the  project  managers  to  be  very 
knowledgeable  in  project  performance  and  possess  the  ability  to  recognize  imme¬ 
diately  when  a  problem  requires  their  attention  (3:365;  1:34-45;  9:21-22;  12).  Such 
skills  can  only  be  exercised  by  someone  who  has  acquired  and  mastered  a  broad 
range  of  analytical  skills,  including  the  corpus  of  concepts  identified  within  the 
field  of  applied  statistics. 
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DEPARTMENT  OF  THE  AIR  FORCE 

AIR  UNIVERSITY 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 
WRIGHT ‘PATTERSON  AIR  FORCE  3ASE  OH  4S<U3-«5*J 


TSS  SNC  (Lt  Lirmenburger./AUTOVOM  785-5  435) 
sumc-  27XX  Systems  Manager  Survey,  SCN  37-77 


8  June  1987 


3  Survey  Particioant 


1.  As  an  officer  in  the  2 7 XX  caraer  fied,  you  have  been  randomly 
«elactad  to  participate  in  the  Air  Force  research  project  that 
may  have  a  significant  impact  on  your  future. 

2.  We  request  that  you  answer  the  questions  in  the  attached  sur¬ 
vey  to  halo  us  gather  data  needed  to  establish  a  profile  of  27XX 
manaaers1  statistical  background  and  the  post-graduation  benefits 
of  statistical  courses  tauaht  to  27XX  students  wno  attend  AFIT's 
Professional  Continuing  education  (PCS)  or  Graduate  Systems 
Management  programs. 

1.  Please  return  your  completed  survey  in  the  enclosed  envelooe 
within  one  week  of  receipt.  All  participants  in  this  survey  will 
remain  anonymous.  Participation  in  the  survey  is  voluntary. 

4.  If  you  are  interested  in  receiving  a  summary  of  the  results 
of  this  research,  please  write  to: 

A FIT /IS  (Lt  Linnenburger ) 

Wright-Patterson  AF3  03  45433-5583 

of  this  research  effort. 


2  Atchs 

1 .  Survey 

2 .  £n”elooe 


We  *na.c'<  you  for  your  suooort 


‘'DAVID  MILLER,  Lt  Coi,  USA? 
Ac  tin  a  *--2ai 

department  of  Mathematics 
and  Computer  Science 


JTMNGTH  -mhough  knowisogs 


27XX  Systems  Manager  Survey 
USAF  Survey  Number  SCN  87-77 
expires:  30  September  1987 

GENERAL  INFORMATION 

The  purpose  of  this  questionnaire  is  to  obtain  data  concerning: 

1.  The  statistical  concepts  27XX  students  can  be  expected  to  have  encountered 
prior  to  entry  into  AFIT’s  Graduate  Program  in  Systems  Management,  and 

2.  The  applicability  of  specific  statistical  technologies  in  the  day-to-day  working 
environment  of  27XX  officers. 

This  data  will  be  used  to  enhance  our  ability  to  create  statistical  courses  that  are 
responsive  to  the  real  world  needs  of  27XX  officers. 

Please  be  assured  that  all  information  you  provide  will  be  held  in  the  strictest 
confidence.  Your  individual  responses  will  NOT  be  provided  to  management  or  to 
any  other  agency.  Feedback  on  the  study’s  results  will  be  presented  to  management 
only  in  terms  of  group  responses  describing  what  the  “typical”  system  manager 
would  say.  In  addition,  when  the  results  of  this  study  are  published,  readers  will 
NOT  be  able  to  identify  specific  individuals  or  work  groups. 

Thank  you  for  your  cooperation  in  participating  in  this  study.  If  you  have 
any  questions,  please  contact  the  researcher  at  the  following  address: 

Lt  Michael  Linnenburger 
AFIT/LS 

Wright-Patterson  AFB,  OH  45433 
Telephone:  AUTOVON  785-5435 


KEYWORDS 


The  following  axe  definitions  of  key  words  that  recur  throughout  the  ques¬ 
tionnaire: 

1.  Supervisor:  The  person  from  whom  you  take  direction. 

2.  Work  Group:  All  persons  who  take  direction  from  you  or  work  with  you. 

INSTRUCTIONS 

This  questionnaire  contains  99  items  (individual  “questions”).  The  items 
must  be  answered  by  filling  in  the  appropriate  response  in  the  space  provided.  If 
for  any  item  you  do  not  find  a  response  that  fits  your  situation  exactly,  mark  the 
one  that  is  the  closest  to  the  way  you  feel. 


Demographics 


The  statements  below  request  information  about  your  military  and  academic  back¬ 
ground  and  current  job. 

1.  What  is  your  rank? 

1.  2nd  Lieutenant 

2.  1st  Lieutenant 

3.  Captain 

2.  What  is  your  current  DUTY  AFSC? 

1.  2721  3.  2711 

2.  2724  4.  2716 


4.  Major 

5.  Lieutenant  Colonel 

6.  Colonel 


3.  What  is  the  highest  academic  degree  you  hold? 

1.  Bachelors  Degree 

2.  Masters  Degree 

3.  Doctorate  Degree 


4.  In  what  field  is  (are)  your  bachelors  degree(s)  (Mark  more  than  one  if  neces¬ 
sary)? 


1.  Arts 

2.  Business 

3.  Engineering 

4.  Sciences 


5.  Management 

6.  Math 

7.  Other  (specify) 


5.  If  you  have  a  Masters  Degree  from  AFIT,  what  education  code  identifies  your 
degree? 


1.  Systems  Management,  1ASY 

j  2.  Engineering  Management,  1AGY 

! 

3.  R&  D  Management,  1APY 

4.  Other  (specify) 

i  6.  To  which  Major  Command  are  you  assigned? 

> 

I 

! 

6.  AFLC 

7.  Space  Command 

8.  Other  (Please  Specify) 


1.  AFSC 

2.  AFCC 

3.  SAC 

4.  TAC 

5.  MAC 


7.  How  would  you  classify  the  job  function  in  which  you  currently  work? 


* 

I 

1.  Program/ Project  manager 

2.  Test  manager 

3.  Logistics  manager 

4.  Financial  manager 

5.  Configuration/Data  manager 

6.  Project  Engineer 


7.  Program  Administration 

8.  Contract  manager 

9.  Production/Manufacturing  man¬ 
ager 

10.  Staff 

11.  Other  (specify) 
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8.  How  many  people  do  you  directly  supervise?  (i.e.,  those  which  you  direct  the 
action  of) 


1. 

0-1 

6. 

16-50 

2. 

2-3 

7. 

51-75 

3. 

4-5 

8. 

76-100 

4. 

6-8 

9. 

101-150 

5. 

9-15 

10 

.  More  than  151 

9.  How  much  experience  do  you  have  as  a  27XX? 

1.  Less  than  6  mos  6.  6  yr  but  less  than  8  yr 

2.  6  mos  but  less  than  1  yr  7.  8  yr  but  less  than  10  yr 

3.  1  yr  but  less  than  2  yr  8.  10  yr  but  less  than  12  yr 

4.  2  yr  but  less  than  4  yr  9.  12  yr  but  less  than  14  yr 

5.  4  yr  but  less  than  6  yr  10.  14  yr  or  more  yrs 


Part  II. 
Job  Tasks 


v'l 


5 


■vH 


ft 


<:■  *:■ 


The  statements  in  this  part  of  the  questionnaire  are  designed  to  help  us 
understand  the  role  you  play  in  making  decisions  or  talcing  actions  in  the  perfor¬ 
mance  of  your  job  as  a  27XX  manager.  Your  responses  should  be  based  on  your 
current  job.  Please  use  the  following  scale  for  your  responses: 


1.  I  PLAY  NO  ROLE  IN  THE  DECISION  MAKING  OR  ACTION 
TAKEN  TO  ACCOMPLISH  THIS  TASK 


2.  I  PLAY  SOME  ROLE  THE  IN  DECISION  MAKING  OR  AC¬ 
TION  TAKEN  TO  ACCOMPLISH  THIS  TASK 


3.  I  PLAY  A  ROLE  ABOUT  HALF  THE  TIME  IN  THE  DECISION 
MAKING  OR  ACTION  TAKEN  TO  ACCOMPLISH  THIS  TASK 


4.  I  PLAY  A  LARGE  ROLE  IN  THE  DECISION  MAKING  OR 
ACTION  TAKEN  TO  ACCOMPLISH  THIS  TASK 


5.  I  AM  THE  ONLY  PLAYER  IN  THE  DECISION  MAKING  OR 
ACTION  TAKEN  TO  ACCOMPLISH  THIS  TASK 


1  (No  Role)  2 


3  (Half)  4 


5  (Sole) 


10.  Approve  or  disapprove  award  fees 


2  3  4  5 


11.  Perform  site  inspections  to  observe  contractor 
or  QAE  performance 


2  3  4  5 


12.  Approve  or  disapprove  inputs  to  Memoranda 
of  Understanding  (MOU)  or  Memoranda  of  Agree¬ 
ments  (MOA) 


13.  Develop  Industrial  Preparedness  (IPP)  plans 


14.  Determine  training  requirements 


15.  Approve  or  disapprove  source  selection 


16.  Approve  or  disapprove  Acquisition  Management 
Reviews 


2  3  4  5 


\V 


1  (No  Role)  2 


3  (Half) 


4 


5  (Sole) 


17.  Recommend  approval  or  disapproval  of  con¬ 
tracts 


IS.  Approve  or  disapprove  Contract  Management 
Reviews 


19.  Prepare/present  formal/informal  briefings 

20.  Analyze  budgeting  requirements 

21.  Analyze  quality  assurance  problems 


22.  Approve  or  disapprove  inputs  to  Five  Year  De¬ 
fense  Plan  (FYDP)  or  Program  Objective  Memoran¬ 
dum  (POM) 


23.  Evaluate  effect  of  funding  cuts  on  programs 


24.  Justify  budget  proposals  to  Program  Review 
Committees 


25.  Coordinate  with  prime  contractor  on  status  of 
subcontracts 


26.  Monitor  fund  allocations 

27.  Determine  status  of  action  items 


28.  Review  fund  expenditures  on  materials,  such  as 
equipment  or  supplies 


29.  Compare  progress  reports  with  established  or 
proposed  schedules 


30.  Develop  budgets  or  budget  estimates 


31.  Approve  or  disapprove  Production  Readiness 
Review  (PRR)  plans 


32.  Analyze  cost  proposals 


1 

2 

1 

3 

1. 

4 

1 

5 

1 

2 

l 

3 

1 

4 

1 

5 

1 

2 

1 

3 

1 

4 

l 

5 

1 

2 

1 

3 

1 

4 

1 

5 

1 

2 

I 

3 

4 

1 

5 

1 

2 

i 

3 

4 

5 

1 

2 

l 

3 

1 

4 

1 

5 

1 

2 

i 

3 

1 

4 

1 

5 

1 

2 

-  i  -  - 

3 

1 

4 

1 

5 

1 

2 

1 

3 

1 

4 

i 

5 

1 

2 

1 

3 

1 

4 

1 

5 

1 

2 

1  _ 

3 

1 

4 

I 

5 

1 

2 

J- ...  _ 

3 

__  1 

4 

1 

5 

1 

l 

2 

1 

3 

1 

4 

5 

1 

1 

2 

1 

3 

4 

1 

5 

1 

2 

3 

4 

5 
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1 


1 


4 


j 


1  (No  Role) 


3  (Half) 


5  (Sole) 


33.  Approve  or  disapprove  management  system  re¬ 
view  meetings 

34.  Analyze  direct  or  indirect  rates 

35.  Validate  requirements 

36.  Analyze  overhead  or  direct  labor  proposals 

37.  Develop  Program  Management  Plans  (PMP) 

38.  Develop  cost  forecasting  models 

39.  Evaluate  Engineering  Change  Proposals  (ECP) 

40.  Develop  overhead  rate  projections 

41.  Develop  target  price  positions 

42.  Develop  Statements  of  Work  (SOW)  or  specifi¬ 
cations 

43.  Analyze  preliminary  or  critical  designs  for  pro- 
ducibility 

44.  Approve  or  disapprove  inputs  to  five-year  plans 

45.  Determine  status  of  milestones 

46.  Approve  or  disapprove  budget  planning  or  re¬ 
view  meetings 

47.  Advise  on  system  matters  during  negotiations 

48.  Review  RFP/RFQs 

49.  Coordinate  with  customers  or  finance  personnel 
on  availability  ot  funds 


1 

_ 1 _ 

2 

1 

_ 1 _ 

3 

1 

4 

i 

5 

1 

2 

1. 

3 

_  1 

4 

_  L  _ 

5 

1 

2 

l 

3 

1 

4 

i 

5 

1 

2 

1 

3 

1  . 

4 

1_ 

5 

1 

2 

1 

3 

1 

4 

1 

5 

1 

2 

_L 

3 

1 

4 

1 

5 

1 

2 

3 

1 

4 

1 

5 

1 

2 

1 

3 

l 

4 

1 

5 

1 

2 

3 

4 

l 

5 

1 

2 

1 

3 

1 

4 

1 

5 

1 

2 
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3 

1 

4 

1 

5 

1 

2 

i 

3 

1 

4 

1 

5 

1 

2 

1 

3 

1__ 

4 

1 

5 

1 

2 

1 

3 

i 

4 

i 

5 

1 

2 

1 

3 

1  . 

4 

1 

5 

1 

2 

3 

1 

4 

1 

5 

1 

2 

3 

4 

5 
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1  (No  Role)  2 


3  (Half) 


4 


5  (Sole) 


50.  Develop  business  strategies  for  acquisitions 

51.  Prepare  Contract  Data  Requirements  Lists 

(CDRL) 

52.  Review  cost  analysis  reports 

53.  Review  PMDs 

54.  Review  proposal  evaluation  reports,  such  as 
technical  evaluations  or  price  evaluations 

55.  Compare  incurred  rates  with  negotiated  rates 

56.  Review  recurring  budget  reports 

57.  Conduct  contractor  progress  review  or  final  re¬ 
view  conferences 

58.  Conduct  trade-off  analyses 

59.  Coordinate  with  contractors  on  warranty  work 

60.  Justify  changes  to  budget  allocations 

61.  Designate  subcontracts  as  major  or  critical 

62.  Monitor  delivery  dates 

63.  Evaluate  C/SCSC  reports 

64.  Assist  negotiation  considerations  for  contrac¬ 
tors’  failure  to  perform 

65.  Evaluate  contractors’  requests  for  reimburse¬ 
ments 

66.  Assist  negotiation  with  contractors  on  modifica¬ 
tion  to  contracts 

67.  Identify  alternate  sources  for  terminated  con¬ 
tractors 
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1 

_ 1 _ 

2 

1 

_ 1 _ 

3 

1 

_ 1 _ 

4 

1 

5 

1 

2 

i 

3 

1 

4 

1 

5 

1 

2 

1 

3 

1 

4 

1 

5 

1 

2 

1 

3 

1 

4 
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5 

1 

2 

1 

3 

l 
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1 

5 

1 

2 

l 

3 

l 

4 

1 

5 

1 

2 

3 

1 

4 

1 

5 

1 

2 
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3 

1 

4 

1 

5 

1 

2 

1 

3 

1 

4 

.  1 

5 

1 

2 

.  1 

3 

__  _  1 

4 

1 

5 

1 

2 

- _ i 

3 

1 

4 

1 

5 

1 

2 

_  .  1 

3 

1 

4 

_  _  1 

5 

1 

2 

-  1 

3 

1 

4 

l 

5 

1 

2 

1 

3 

1 

4 

1 

5 

1 

2 

1 

3 

1 

4 

5 

1 

2 

3 

1 

4 

1 

5 

1 

1 

2 

1 

3 

1 

4 

5 

1 

2 

3 

4 

5 

1  (No  Role)  2 


3  (Half) 


4 


5  (Sole) 


68.  Assist  negotiation  considerations  for  specifica¬ 
tion  deviations  or  waivers 

69.  Assist  negotiation  of  spare  paxts  prices  or  deliv¬ 
ery  schedules 


70.  Assist  negotiation  with  contractor  on  contract 
change  proposals 


PCE  Courses 


The  statements  in  this  part  of  the  questionnaire  are  designed  to  help  us 
understand  the  role  the  following  courses  play  in  decision  making  or  action  taking 
place  during  the  performance  of  your  job  as  a  27XX  manager. 

Tell  us  whether  or  not  you  completed  the  course.  Then  tell  us  whether  you 
have  been  able  to  use  the  technologies  covered  in  the  course  in  your  current  job. 

Complete  the  Used  column  only  if  Completed  column  is  YES. 

Used 


71.  Introduction  to  Systems 
Command  Acquisition  Manage¬ 
ment  (SAS  001) 

72.  Introduction  to  Acquisition 
Management 

(AFIT  SYS  100) 

73.  Acquisition  Planning  and 
Analysis  (AFIT  SYS  200) 

74.  Fundamentals  of  Acquisition 
Management 

(AFIT  SYS  123) 

75.  Systems  Program  Manage¬ 
ment  (AFIT  SYS  223) 

76.  Introduction  to  Configuration 
Management 

(AFIT  SYS  028) 

77.  Advanced  Configuration  Man¬ 
agement  (AFIT  SYS  128) 

78.  Applied  Configuration  Man¬ 
agement  (AFIT  SYS  228) 

79.  Intermediate  Program  Man¬ 
agement  (AFIT  SYS  400) 


.  YES 

NO 

YES 

NO 

.  YES 

NO 

YES 

NO 

.  YES 

NO 

YES 

NO 

.  YES 

NO 

YES 

NO 

.  YES 

NO 

YES 

NO 

.  YES 

NO 

YES 

NO 

.  YES 

NO 

YES 

NO 

.  YES 

NO 

YES 

NO 

.  YES 

NO 

YES 

NO 

80.  Test  and  Evaluation  Manage¬ 
ment  (AFIT  SYS  229) 

.  YES 

NO 

YES 

NO 

81.  Reliability  and  Maintainabil¬ 
ity  in  Systems  Acquisition  (AFIT  . 
QMT  335) 

.  YES 

NO 

YES 

NO 

82.  Contractor  Performance  Mea¬ 
surement  (DSMC) 

.  YES 

NO 

YES 

NO 

83.  Contract  Finance  for  Program 
Managers  (DSMC) 

.  YES 

NO 

YES 

NO 

84.  Contract  Overhead  Monitor- 
ship  (AFIT  QMT  355) 

.  YES 

NO 

YES 

NO 

85.  Cost  Improvement  Analysis 
(AFIT  QMT  180) 

.  YES 

NO 

YES 

NO 

86.  Quantitative  Techniques  for 
Cost  &  Price  Analysis 
(AFIT  QMT  345) 

.  YES 

NO 

YES 

NO 

87.  Introduction  to  Life  Cycle 
Costing  (AFIT  QMT  353) 

.  YES 

NO 

YES 

NO 

88.  Cost/Schedule  Control  System 
Criteria 

(AFIT  SYS  362) 

.  YES 

NO 

YES 

NO 

89.  Surveillance  of  Cost  & 

Schedule  Control  Systems  Criteria 

.  YES 

NO 

YES 

NO 

(AFIT  SYS  361) 


90.  Advanced  Cost  &  Economic 
Analysis  (AFIT  QMT  551) 


YES 


NO 


YES 


NO 


Part  IV. 

Academic  Courses 

The  statements  in  this  part  of  the  questionnaire  axe  designed  to  help  us 
understand  the  role  the  following  courses  play  in  decision  making  or  action  taking 
place  during  the  performance  of  your  job  as  a  27XX  manager. 

Tell  us  whether  or  not  you  completed  the  course.  Then,  tell  us  whether  you 
have  been  able  to  use  the  technologies  covered  in  the  course  in  your  current  job. 


Complete  the  Used  column  only  if  Completed  column  is  YES. 


Completed 

Used 

91.  Applied  Probability 

...  YES 

NO 

YES 

NO 

92.  Applied  Statistics  (Descriptive 
Statistics  and  Hypothesis  Test-  . . . 
ing) 

...  YES 

NO 

YES 

NO 

93.  Applied  Statistics  (Regression 
and  Analysis  of  Variance) 

...  YES 

NO 

YES 

NO 

94.  Time  Series  Analysis 

...  YES 

NO 

YES 

NO 

95.  Multivarate  Analysis  (Fac¬ 
tor  Analysis,  Canonical  correla-  . . . 
tion,  MANOVA) 

...  YES 

NO 

YES 

NO 

96.  Categorical  Data  Analysis 
(Contingency  Tables) 

...  YES 

NO 

YES 

NO 

97.  Applied  Decision  Analysis 
(Baysian  Statistics) 

...  YES 

NO 

YES 

NO 

98.  Design  of  Experiments 

...  YES 

NO 

YES 

NO 

99.  Non-Parametric  Statistics 

..  YES 

NO 

YES 

NO 

Thank  you  for  completing  this  survey.  We  invite  any  comments  you  have  on 
the  subject  of  this  survey  and/or  the  survey  itself.  Space  for  comments  has  been 
provided  on  the  following  page. 
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Appendix  B.  Survey  Comments 


NOTE:  The  use  of  the  symbol  ^  is  to  indicate  the  start  of  a  selected  indi¬ 
viduals  comment. 

^ —  Application  of  management  principles  and  sense  of  teamwork  gained  in  the 
operational  career  fields  is  important  to  those  of  us  who  career  broaden  into  program 
management.  Descision  making  techniques  learned  in  SOS  and  ACSC  also  are  of  great 
importance  to  the  program  manager. 

f —  It  makes  a  big  difference  if  you  have  had  some  experience  before  taking  an 
AFIT  course. 

f —  I  see  little  use  for  an  elaborate  statistical  background.  In  the  real  world,  that’s 
not  how  decisions  are  made. 

f —  I  have  taken  18hrs  on  MBA  and  use  the  statistics  and  management  everyday.  I 
would  be  more  then  willing  to  take  the  PCE  courses  you  outlined.  I  have  asked  for  DSMC 
and  been  recommended  by  my  bosses  for  the  last  four  years.  But  as  you  are  aware  that 
is  a  hard  school  to  get  into. 

H —  I  have  had  no  use  for  statistics  in  my  job. 

f —  I’ve  used  the  thought  processes  taught  in  economic  analysis  in  my  decision 
making  processes.  Decisions  in  acquisition  management  must  also  consider  utility  and 
marginal  value.  These  can  be  considered  in  theory  using  empirical  analysis;  however, 
the  program  manager  doesn’t  have  the  resources  or  data  bases,  normally,  to  do  in-depth 
empirical  studies  to  determine  a  decision.  This  leaves  the  manager  with  following  his 
intuitive  judgement.  Some  people  have  a  good  intuition,  some  don’t.  Generally  those  who 
can  do  an  excellent  empirical  analysis  can’t  for  the  life  of  them  understand  the  underlying 
issues  that  could  steer  a  decision  away  from  the  numbers.  Conversely,  those  who  have 
a  good  intuition  can’t  understand  the  empirical  approach  when  confronted  with  sound 
quantitative  facts  to  go  a  certain  way  with  a  decision,  policy  or  contrast. 

f —  Part  IV.  —  More  importantly  than  understanding  the  technical  details  of  vari¬ 
ous  statistical  methods,  the  greatest  value  of  statistical  courses  (ESP,  Applied  Statistics) 
to  me  as  an  acquisition  manager  has  been  learning  to  question  input  parameters,  assump¬ 
tions  and  output  sensitivities! 

1 —  There  are  a  lot  of  requirements  from  my  job  that  could  not  be  filled  solely  with 
AFIT  courses.  Some  AFIT  courses  do  not  prepare  you  for  real  life  experiences.  Most  of 
the  emphasis  is  on  ideal  situations  and  not  on  reality.  AFIT  usually  focuses  on  major 
programs  and  SPOs  without  considering  small  programs  and  laboratory  efforts. 

5 —  There  must  be  a  way/course  (i.g.,  2  to  3  months)  to  teach  an  individual  new  to 
the  acquisition  business  the  big  picture  without  being  fed  piece  meal  in  two  week  periods 
spread  over  2  to  3  years! 

We  have  a  concentrated  training  program  for  pilots,  engineers,  maint  officers,  missile 


officers,  but  not  AFSC  SPO  acquisition  officers.  Therefore  the  people  developing  the 
operational  equipment  make  mistakes  at  everyone  else’s  expense! 

% —  In  my  present  job,  I  use  very  little  math.  I  feel  that  an  introductory  -  very 
basic  -  class  on  statistical/mathematical  techniques  could  be  useful  to  me.  However,  a 
highly  technical  class  would  be  a  waste  of  time. 

f —  Your  study  has  made  me  realize  how  little  that  I  did  know  and  more  importantly 
how  much  I  don’t  do.  My  background  is  ops  -  and  I  believe  that  that  Education  speaks 
for  itself  and  using  that  background  I  have  been  able  to  interpret  AFSC  into  English  (and 
stay  productively  employed  for  40  hrs  per  week).  Hopefully  after  AFIT  sees  fit  to  schedule 
more  of  your  classes  I  can  truly  become  a  productive  member  of  society. 

f —  Statistics  are  misapplied  so  often  that  everyone  should  know  how  statistics 
works.  Specifically,  in  evaluating  weapon  system  performance,  it  is  critical  to  recognize 
the  proper  distribution  to  use,  sample  size,  and  where  varying  Test  Conditions  invalidate 
test  results.  Unfortunately,  a  high  percentage  of  Air  Force  Testing  appears  to  not  be  a  good 
sample.  Cost  of  assets  and  the  numerous  variables  needed  to  duplicate  the  operational 
environment  (which  create  the  need  for  large  sample  sizes)  often  causes  the  evaluation  of 
a  weapon  system’s  performance  to  degenerate  into  subjective  feelings  and  guesses  rather 
than  objective  evaluation  based  on  an  adequate  sample  size  in  testing. 

1 —  The  best  course  that  I  have  taken  for  program  management  was  SYS  200.  I 
thought  it  was  very  worthwhile  and  practical.  If  other  courses  could  be  structures  more 
like  that,  I  believe  they  would  be  of  greater  value. 

f —  My  PhD  in  Economics  with  emphasis  in  econometric  has  provided  me  with 
the  education  base  to  perform  cost  estimating  C/SCSC,  and  budgetary  functions.  Prior 
Air  Staff  experience  in  DCS/PR  has  provided  an  opportunity  to  understand  the  internal 
workings  of  the  Planning,  Programming,  and  Budgeting  process  necessary  building  a  good 
working  relationship  between  Program  Control  and  Command  and  Air  Staff  counterparts. 

5 —  As  a  2716  with  almost  20  years  service  it  is  difficult  to  respond  to  some  of 
the  questions  regarding  utility  of  various  courses  in  my  current  job.  Obviously  many  of 
the  courses  in  III  and  IV  have  been  directly  used  throughout  my  career.  They  may  not 
enter  directly  into  my  job  responsibility  today  but  formed  the  basis  for  my  being  able  to 
progress  from  a  specific  technical  responsibility  to  a  broadened  management  role  at  the 
headquarters  (MAJCOM  or  Air  Force). 

5 —  Statistics  is  of  use  to  me  only  as  it  provides  the  ability  to  critically  evaluate 
other’s  use  of  it. 

5 —  The  questions  in  part  II  are  highly  skewed  to  contracts  and  cost  estimating. 
There  is  no  mention  of  the  value  of  a  background  in  statistics  . .  .It’s  particularly  helpful 
in  understanding  a  contractors  analysis,  but  is  not  used  as  a  desk  tool  for  me.  I  need  the 
understanding,  not  the  analytical  skill. 

f —  Applied  probability  and  applied  statistics  are  probably  the  most  useful.  I  had 
the  opportunity  to  do  some  number  crunching  in  defining  the  reliabililty  requirements  for 
the  . .  .program. 

It’s  very  difficult  to  communicate  reliability  concepts  with  an  untrained  person.  The 
emphasis  now  is  on  reliability  but  we  have  very  few  trained  managers  in  the  Air  Force 


today  who  really  understand  those  concepts  and  using  applied  statistics  in  defining  test 
requirements. 

In  short,  we  need  to  develop  a  short  course  for  the  senior  managers  as  well  as 
workers  to  bring  about  a  better  understanding  of  those  concepts.  Why?  Because  of  a 
lack  of  understanding  we  sometimes  ask  for  things  that  are  impossible  or  we  end  up  with 
farless  than  optimum  cost/benefit  ratio. 

You  should  do  a  sample-test  survey  on  reliability  concepts.  Thank  you. 

f-  The  only  time  I  had  any  use  of  the  AFIT  stat  course  was  at  a  design  review. 
We  were  looking  at  RMA  formulas  that  the  contracts  was  using. 

If —  My  experiences  as  a  program  manager  2724  and  2711  have  not  required  the  use 
of  any  statistical  analysis  technics.  My  brief  exposure  to  statistics  at  . .  .where  I  obtained 
my  MBA,  does  not  provide  a  clue  to  using  statistical  analysis  techniques. 

f —  Program  Managers  need  training  in:  Program  Management,  Configuration 
Mgmt  (ECP/CCP  Processing),  Financial  Mgmt  (Budget,  Expenditure/Obligation),  Test 
Mgmt  (Facilities,  Coordination),  Contract  Mgmt  (Pricing  Evaluation,  Negotiation,  CLIN 
Structure/Pricing),  Logistics  Mgmt  (Outyear  Requirement,  Deployment  Scheduling,  War¬ 
ranty  Structure). 

f - Even  though  I  have  not  directly  used  statistics  (#92-93)  and  analysis  (#96),  I 

feel  they  are  overall  beneficial  -  I  lack  experience  in:  "situations”  to  use  them  and  "when” 
to  use  them  -  not  really  in  "how  "  to  use  them. 

-Also,  I  think  27xx’s  should  have  these  AFIT  classes  more  available  to  them.  At 
my  base,  no  one  even  Knows  they  exist  or  that  they  are  available  to  us.  They  should  be 
available  as  short  courses? 

H —  All  PMs  and  Engineers  NEED  to  understand  statistical  analysis  as  it  applied 
to  reliability.  I  don’t  know  much  but  I’ve  seen  data  misrepresented. 

f —  I  have  an  engineering  Phd  from  . . .  with  substantial  statistical  background/prob- 
ability.  I  regard  statistics  as  a  key  discipline  for  all  officers  doing  program  management 
for  Systems  Command. 

% —  I  had  QMT  576  which  was  50%  statistics.  Used  somewhat  as  R&M  engineer 
but  use  no  statistics  in  current  2724  position.  Also  had  government  course  in  statistical 
quality  control  -  used  in  supervisory  QA  position  and  now  when  dealing  with  QA  issues. 
Statistics  courses  would  be  low  in  my  priority  for  anyone  in  27XX  career  field. 

t —  I  feel  statistics  has  been  very  useful  in  some  aspects  of  my  job  because  I  acquire 
missile  warning  systems  and  we  do  have  algorithm  analysis.  However,  this  work  has  not 
been  hands  on.  It  has  been  more  in  an  advisory  program  management  role  (  mostly 
because  people  know  I  do  have  this  background). 

I  feel  for  the  average  27xx  who  does  not  work  with  direct  statistical  systems  on  a 
daily  basis,  that  courses  oriented  more  toward  the  ops  research  side  of  statistics  could  be 
a  lot  more  useful  than  "pure”  statistics  courses. 

Personally,  however,  I  would  love  to  see  you  offer  more  probability  and  statistics 
courses. 


5 —  The  use  of  most  courses  has  been  in  general,  permitting  a  more  thorough 
understanding  of  contractor’s  reports,  studies,  etc. 

While  few  AFIT  courses  were  checked  local  in-house  courses  and  DSMC  courses 
also  provide  good  source. 

^ —  As  part  of  my  Grad  Log  Degree  I  tool  statistical  and  QDM  courses.  They  are 
useful  in  my  current  job. 

f —  I  have  had  statistics  and  decision  analysis  in  my  undergrad  and  masters  -  I 
have  used  them  -  not  extensively  but  when  I  do  I’m  glad  I  know  the  material. 

Courses  like  SYS  200  are  invaluable  at  the  one  yr  point  -  we  need  more  slots  for 
these  courses. 

f —  Although  I  do  not  currently  use  these  courses  in  my  job,  there  are  many  times 
when  the  basics  of  statistics  and  probability  have  come  into  play.  Setting  test  criteria 
often  includes  confidence  levels.  I  do  not  do  these  calculations,  but  must  understand  them 
to  make  pass/fail  decisions  on  test  results. 

f —  I  like  your  purpose  statement  and  have  significant  background  and  knowledge 
that  would  be  of  value  to  you.  For  instance,  I  have  been  a  test  engineer  doing  reliability 
testing  with  confidence  levels,  etc. 


However,  the  real  question  is  the  real  world  needs  of  27xx’s  for  training  in  statistics 
—  absolutely!  Some  needs  are  obvious,  such  as  my  duties  with  reliability  testing,  where 
detailed  knowledge  is  required.  At  that  same  time,  the  design  engineers,  technical  man¬ 
agers,  program  managers,  and  entire  chain  of  command  found  themselves  in  the  position  of 
needing  and/or  wanting  to  understand  the  real  significance  of  the  test  results  -  often  with 
little  background  in  statistics.  Also,  often  with  a  great  fear  of  statistics  and  statisticians! 

5 —  . .  .In  considering  the  development  of  courses  please  stress  application.  Life  out 
here  is  much  different  from  academia  and  the  learning  of  knowledge  should  be  approached 
in  a  manner  much  different  than  typical  courses.  Application  of  probability  and  statistics 
to  budget  (BES/POM),  Risk  Analysis/Abatement,  cost  analysis,  as  well  as  more  typical 
production  related  analyses  would  be  helpful.  Particular  attention  must  be  paid  to  val¬ 
idating  assumptions  and  proper  use  of  tools  available  ...use  existing  real  life  data  from 
AFSC  and  various  SPO’s  . .  .wide  range  of  PC  tools. 
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Appendix  C.  Statistical  Tables 
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Figure  7:  Ql  Current  Rank;  Entire  Survey  Population 
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Figure  8:  Current  Duty  AFSC;  Entire  Survey  Population 


66 


Q3 


HIGHEST  ACADEMIC  DEGREE  OBTAINED  FREQ 


BACHELORS  DEGREE 
MASTERS  DEGREE 
DOCTORATE  DEGREE 


I******************  182 

j  ********************************  320 
I*  8 


-+ - + - + - + - + - +-■ 

50  100  150  200  250  300 


Figure  9:  Q3  Highest  Academic  Degree  Obtained;  Entire  Survey  Population 
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Figure  10:  Q4  Primary  Academic  Background;  Entire  Survey  Population 
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Figure  11:  Q5  Education  Code  for  MS  from  AFIT;  Entire  Survey  Population 
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Figure  12:  Q5b  AFIT  Graduate  or  not;  Entire  Survey  Population 
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Figure  13:  Q6  Major  Command  Assigned  to;  Entire  Survey  Population 
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Figure  14:  Q7  Current  Job  Function;  Entire  Survey  Population 
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Figure  15:  Q8  Number  of  People  Supervised;  Entire  Survey  Population 
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Figure  16:  Q9  Experience  as  27XX;  Entire  Survey  Population 
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Figure  17:  Ql  Current  Rank;  All  Potential  GSM  Students  from  27xx  AFSC 
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Figure  18:  Q2  Current  Duty  AFSC;  All  Potential  GSM  Students  from  27xx  AFSC 
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Figure  19:  Q3  Highest  Academic  Degree  Obtained;  All  Potential  GSM  Students 
from  27xx  AFSC 
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Figure  20:  Q4  Primary  Academic  Background;  All  Potential  GSM  Students  from 
27xx  AFSC 
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Figure  21:  Q6  Major  Command  Assigned  to;  All  Potential  GSM  Students  from 
27xx  AFSC 
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Figure  22:  Q7  Current  Job  Function;  All  Potential  GSM  Students  from  27xx 
AFSC 
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Figure  23:  Q8  Number  of  People  Supervised;  All  Potential  GSM  Students  from 
27xx  AFSC 


! 

I 


Q9 

EXPERIENCE  AS  27XX 

1 

FREQ 

6  MOS  BUT  <  1  YR 

i 

| ******** 

42 

1  YR  BUT  <  2  YR 

| ********* 

44 

2  YR  BUT  <  4  YR 

| ********************** 

110 

4  YR  BUT  <  6  YR 

| ******* 

37 

6  YR  BUT  <  8  YR 

| ******* 

33 

8  YR  BUT  <  10  YR 

j  **** 

20 

10  YR  BUT  <  12  Y 

|  * 

5 

12  YR  BUT  <  14  Y 

1 

1 

>  14  YR 

1 

1 

1 

-  + - + - + - + - +  _ 

20  40  60  80  100 


Figure  24:  Q9  Experience  as  27XX;  All  Potential  GSM  Students  from  27xx  AFSC 
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PCE  Course  Contingency  Tables  Summary 


Table  12:  Summary  of  PCE  Course  Data 


Ques¬ 
tion  # 

Course 

Title 

Reject 

H0 

Num¬ 

ber 

of 

YES/ 

YES 

Per¬ 

cent 

of 

YES/ 

YES 

Num¬ 

ber 

in 

Table 

Signif¬ 

icant 

YES/ 

NO 

Q71 

SAS  001 

12.100 

128 

134 

Q72 

SYS  100 

30.197 

118 

130 

Q73 

SYS  200 

28.766 

YES 

109 

94.78 

115 

YES 

Q74 

SYS  123 

17.255 
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16 

72.73 

22 

YES 

Q75 
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YES 

27 

79.41 

34 

YES 
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38 
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48 
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32 
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21 
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6 

NO 
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10 
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NO 
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NO 

Academic  Course  Contingency  Tables  Summary 


i  Table  13:  Summary  of  Academic  Course  Data 

I 
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Reject 
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YES/ 
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Table 
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icant 

YES/ 

NO 

Q91 
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NO 
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NO 
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NO 
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ries  Anal 

0.194 

NO 

4 

9.09 

44 

NO 

Q95 

Multivar 

Anal 

0.793 

NO 

7 

21.21 

33 

NO 

Q96 

Categ 

Data  Anal 

0.058 

NO 

8 

20.00 

40 

NO 

Q97 

Applied 
Desc  Anal 

3.554 

NO 

19 

34.55 

55 

NO 

Q98 

Design  of 
Exp 

1.853 

NO 

15 

31.25 

48 

NO 

Q99 

Non- 

Par  ametric 
Stat 

3.500 

NO 

7 

20.00 

35 

NO 

Task  Function  ANOVA 


Table  14:  Means  of  the  Six  Task  Functions;  Entire  Survey  Population 


VARIABLE 

N 

N  MISSING 

MEAN 

STANDARD 

DEVIATION 

FUNC 

515 

- (jKUUr- 1 - 

0 

1.801 

0.794 

FUNC 

515 

- vjKUUraz - 

0 

2.311 

0.968 

FUNC 

515 

— -  - 

0 

2.574 

1.097 

FUNC 

515 

0 

2.127 

0.903 

FUNC 

515 

0 

1.730 

0.685 

FUNC 

515 

0 

2.321 

0.838 

Table  15:  Analysis  of  Variance  Procedure;  Dependent  Variable:  FUNC 

ANALYSIS  OF  VARIANCE  PROCEDURE 
DEPENDENT  VARIABLE:  FUNC 


SOURCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

MODEL 

5 

274.48787899 

54.89757580 

ERROR 

3084 

2445.23082545 

0.79287640 

CORRECTED  TOTAL 

3089 

2719.71870444 

MODEL  F  * 

69.24 

PR  >  F  *  0.0 

Table  16:  Tukey’s  Studentized  Range  (HSD)  Test  for  Variable:  FUNC 

ANALYSIS  OF  VARIANCE  PROCEDURE 

TUKEY’S  STUDENTIZED  RANGE  (HSD)  TEST  FOR  VARIABLE:  FUNC 
NOTE:  THIS  TEST  CONTROLS  THE  TYPE  I  EXPERIMENT  WISE  ERROR  RATE 

ALPHA-0.05  CONFIDENCE-O . 95  DF-3084  MSE-0. 792876 

CRITICAL  VALUE  OF  STUDENTIZED  RANGE-4.033 
MINIMUM  SIGNIFICANT  DIFFERENCE- . 15823 

COMPARISONS  SIGNIFICANT  AT  THE  0.05  LEVEL  ARE  INDICATED  BY  »***» 


SIMULTANEOUS 

SIMULTANEOUS 

LOWER 

DIFFERENCE 

UPPER 

GROUP 

CONFIDENCE 

BETWEEN 

CONFIDENCE 

COMPARISON 

LIMIT 

MEANS 

LIMIT 

3 

-  6 

0.09437 

0.25260 

0.41083 

*** 

3 

-  2 

0.10468 

0.26291 

0.42114 

*** 

3 

-  4 

0.28866 

0.44689 

0.60512 

*** 

3 

-  1 

0.61471 

0.77294 

0.93118 

*** 

3 

-  5 

0.68505 

0.84328 

1.00151 

*  *  * 

6 

-  3 

-0.41083 

-0.25260 

-0.09437 

*** 

6 

-  2 

-0.14792 

0.01032 

0.16855 

6 

-  4 

0.03606 

0.19430 

0.35253 

*** 

6 

-  1 

0.36212 

0.52035 

0.67858 

*** 

6 

-  5 

0.43245 

0.59068 

0.74891 

*** 

2 

-  3 

-0.42114 

-0.26291 

-0.10468 

*** 

2 

-  6 

-0.16855 

-0.01032 

0.14792 

2 

-  4 

0.02575 

0.18398 

0.34221 

*** 

2 

-  1 

0.35180 

0.51003 

0.66826 

*** 

2 

-  5 

0.42214 

0.58037 

0.73860 

*** 

4 

-  3 

-0.60512 

-0.44689 

-0.28866 

*** 

4 

-  6 

-0.35253 

-0.19430 

-0.03606 

*** 

4 

-  2 

-0.34221 

-0.18398 

-0.02575 

4 

-  1 

0.16782 

0.32605 

0.48428 

*** 

4 

-  5 

0.23815 

0.39639 

0.55462 

*** 

1 

-  3 

-0.93118 

-0.77294 

-0.61471 

*** 

1 

-  6 

-0.67L58 

-0.52035 

-0.36212 

*** 

1 

-  2 

-0.66826 

-0.51003 

-0.35180 

*** 

1 

-  4 

-0.48428 

-0.32605 

-0.16782 

*** 

1 

-  5 

-0.08790 

0 . 07033 

0.22857 

5 

-  3 

-1.00151 

-0.84328 

-0.68505 

*  +  * 

5 

-  6 

-0.74891 

-0.59068 

-0.43245 

*** 

5 

-  2 

-0.73860 

-0.58037 

-0.42214 

*** 

5 

-  4 

-0.55462 

-0.39639 

-0.23815 

*** 

5 

-  1 

-0.22857 

-0.07033 

0.08790 

Task  Item,  Reliability 


Table  17:  Coefficient  Alpha  and  Standardized  Item  Alpha 

COEFFICIENT  ALPHA 
ALPHA  C0L1 

R0W1  0 . 9778 

STANDARDIZED  ITEM  ALPHA 
ALPHA  C0L1 

R0W1  0.9779 


Table  18:  Scale  Statistics  with  Item  Deleted 


ID  STATS 

MEAN 

STD  DEV 

VARIANCE 

CORREL 

SMC 

ALPHA 

Q10 

127.2136 

47.1839 

2226.3 

0.5538 

0.4761 

0 . 9776 

QU 

126.8718 

47.1099 

2219.3 

0.5369 

0.5397 

0 . 9776 

Q12 

126.1650 

47.0115 

2210.1 

0.6168 

0.5402 

0.9775 

qi3 

127.7495 

47.5360 

2259.7 

0.4168 

0.4069 

0 . 9778 

qi4 

126.8233 

47.2704 

2234.5 

0.4171 

0.3156 

0 . 9779 

qis 

127.1087 

47.1541 

2223.5 

0.5670 

0 . 5344 

0 . 9776 

qi6 

127.0039 

47.0925 

2217.7 

0 . 5843 

0.5424 

0 . 9775 

qi7 

126.7981 

46.9261 

2202.1 

0.7115 

0.6562 

0.9773 

qis 

127.1262 

47 . 0084 

2209.8 

0 . 6837 

0.6370 

0.9773 

qi9 

125.2330 

47.2060 

2228.4 

0.5275 

0.4829 

0 . 9776 

q20 

125.9903 

46.8758 

2197.3 

0.7024 

0.7825 

0.9773 

q2i 

126.9107 

47.0609 

2214.7 

0.6292 

0.5996 

0 . 9774 

q22 

126.5573 

47.0683 

2215.4 

0.5278 

0.7277 

0.9777 

q23 

126.1650 

46.9359 

2203.0 

0.6043 

0.7036 

0.9775 

Q24 

126.6117 

46.9812 

2207.2 

0.5778 

0.6872 

0.9776 

Q25 

126.5592 

46.8349 

2193.5 

0.6897 

0.6508 

0 . 9773 

Q26 

126.2155 

46.7987 

2190.1 

0.7035 

0.7604 

0.9773 

Q27 

125.5728 

47.0799 

2216.5 

0.5751 

0.5221 

0.9776 

Q28 

126.5961 

46.9255 

2202.0 

0.6612 

0.6471 

0 . 9774 

q29 

125.8388 

46.9074 

2200.3 

0.6869 

0.6642 

0.9773 

q30 

126.1825 

46.8095 

2191.1 

0.7070 

0 . 7449 

0.9773 

q3i 

127.4214 

47 . 2845 

2235.8 

0.5408 

0.5430 

0 . 9776 

q32 

126.4874 

46.8321 

2193.2 

0.7283 

0.6891 

0.9772 

q33 

126.8854 

46.8923 

2198.9 

0.7143 

0.6229 

0.9773 

q34 

127.4272 

47.2522 

2232.8 

0.6057 

0.6970 

0 . 9775 

q35 

126.4563 

47.0192 

2210.8 

0.5907 

0.4737 

0.9775 

q36 

127.3049 

47 . 1774 

2225.7 

0.6063 

0.6501 

0.9775 

q37 

126.6155 

46.8799 

2197.7 

0.6723 

0.5940 

0 . 9774 

q38 

127.3689 

47.2422 

2231.8 

0.5427 

0.5601 

0.9776 

q39 

126.5243 

46.9496 

2204.3 

0.6262 

0.6635 

0.9775 

q40 

127.6272 

47.3762 

2244.5 

0.5551 

0.6974 

0 . 9776 

q4i 

127.3515 

47.0785 

2216.4 

0.6699 

0 . 6488 

0 . 9774 

q42 

126.2311 

46.8084 

2191.0 

0.6925 

0.7481 

0.9773 

q43 

127.0738 

47 . 0948 

2217.9 

0.6079 

0 . 5477 

0.9775 

q44 

126.7670 

46.9602 

2205.3 

0 . 6333 

0.7222 

0 . 9774 

q45 

126.1107 

46.8585 

2195.7 

0.7080 

0.6745 

0 . 9773 

q46 

126.8233 

46.8985 

2199.5 

0.6988 

0.7084 

0 . 9773 

q47 

126.7961 

46.8104 

2191.2 

0.7543 

0.7279 

0 . 9772 

q48 

126.5126 

46.8345 

2193.5 

0.7395 

0.7162 

0 . 9772 

q49 

126.3922 

46.7900 

2189.3 

0.7248 

0.7194 

0.9772 

qso 

126.5437 

46.8709 

2196.9 

0.6943 

0.6328 

0 . 9773 

qsi 

126 . 6777 

46.9260 

2202 . 1 

0.6566 

0.6987 

0 . 9774 

q52 

126.7786 

46.8953 

2199.2 

0.7110 

0.6780 

0 . 9773 

q53 

126.2505 

47.0557 

2214.2 

0.5350 

0 . 5467 

0 . 9777 

q54 

126.5689 

46.8022 

2190.4 

0.7690 

0.6956 

0 . 9772 

qss 

127.3825 

47.1642 

2224.5 

0.6407 

0.6367 

0 . 9775 

q56 

126.9126 

47.0175 

2210.6 

0.6372 

0.6945 

0 . 9774 

q57 

126.6447 

46.7858 

2188.9 

0.7312 

0 . 6849 

0.9772 

qss 

126.9087 

46.9873 

2207.8 

0.6871 

0 . 6074 

0 . 9773 

q59 

127.3126 

47.1483 

2223.0 

0.6243 

0.5911 

0.9775 

q60 

126.6214 

46.8197 

2192.1 

0.7226 

0.7601 

0 . 9773 

q6i 

127.5010 

47.2530 

2232.8 

0.5986 

0.5661 

0.9775 

q62 

126.3767 

46.8084 

2191.0 

0.6892 

0.6301 

0.9773 

q63 

127.1456 

47.1429 

2222.5 

0.5434 

0.5608 

0 . 9776 

q64 

127.1087 

46.9594 

2205.2 

0.7106 

0.7098 

0 . 9773 

q65 

127.1262 

46.9484 

2204.1 

0.7244 

0 . 6843 

0 . 9773 

q66 

126.7883 

46.7953 

2189.8 

0.7676 

0.8243 

0.9772 

q67 

127.4854 

47.2583 

2233.4 

0.5570 

0.4851 

0.9776 

q68 

126.9864 

46.9706 

2206.2 

0.6602 

0.7021 

0 . 9774 

q69 

127.3670 

47.1537 

2223.5 

0.6162 

0.6095 

0.9775 

q70 

126.8194 

46 . 8347 

2193.5 

0.7377 

0.7994 

0 . 9772 
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